Optimization in Machine Learning Based Applications

Abstract

The use of Machine Learning (ML) can be found in almost all fields of science, technology, arts newlineand commerce. ML is used for designing trained models based on experience and data. They are newlinemathematical models that involve applications of few major branches of mathematics such as newlinestatistics, probability, linear algebra and optimization. These models basically help in prediction and newlinedecision making. Many researchers are trying to build ML models which are effective and efficient newlinein terms of accuracy, time complexity and other performance metrics for solving real world newlineproblems. While designing such models, one must have to consider the environments where these newlinemodels are trained. These environments mainly affect the optimality of the internal components of newlineML models. One of such components, we have broadly studied in this thesis work is the set hyperparameters of ML models. The performance of an ML model is highly dependent on these hyperparameters. In a dynamic environment where training data grows over time, it is very difficult to set newlinethe optimal values of these hyper-parameters because their optimal values keep getting shifted at newlinevarious time instances. This also demands continuous adaptive optimization techniques to optimize newlinethese hyper-parameters. In this thesis work, we have proposed various generalized dynamic newlineoptimization frameworks to build ML models which are robust enough to adapt the changing newlineenvironments of real world. The validation of these frameworks has been done by using them to newlineoptimize the hyper-parameters of Support Vector Machine (SVM) in dynamic environment. newlineFurther, we have also proposed a novel adaptive knowledge-based optimization technique to build newlineeffective and efficient Intrusion Detection System (IDS) in dynamic environment by properly newlineoptimizing the hyper-parameters of SVM. newlineThe proposed frameworks for tuning hyper-parameters of ML models in this thesis work, uses newlinemetaheuristic optimization algorithms such as Moth-Flame Optimization (MFO) and Particle newlineSwarm Optimization (PSO) as the b