An Evolutionary Optimization Algorithm for Automated Classical Machine Learning

Authors

Leila Zahedi, Florida International UniversityFollow

Document Type

Dissertation

Degree

Doctor of Philosophy (PhD)

Major/Program

Computer Science

First Advisor's Name

Dr. M. Hadi Amini

First Advisor's Committee Title

Major advisor

Second Advisor's Name

Dr. S. S. Iyengar

Second Advisor's Committee Title

Committee member

Third Advisor's Name

Dr. Nagarajan Prabakar

Third Advisor's Committee Title

Committee member

Fourth Advisor's Name

Dr. Leonardo Bobadilla

Fourth Advisor's Committee Title

Committee member

Fifth Advisor's Name

Dr. Shabnam Rezapour

Fifth Advisor's Committee Title

Committee member

Keywords

AutoML, Hyperparameter tuning, Feature Selection, Machine Learning, Evolutionary optimization algorithm

Date of Defense

6-29-2022

Abstract

Machine learning is an evolving branch of computational algorithms that allow computers to learn from experiences, make predictions, and solve different problems without being explicitly programmed. However, building a useful machine learning model is a challenging process, requiring human expertise to perform various proper tasks and ensure that the machine learning's primary objective --determining the best and most predictive model-- is achieved. These tasks include pre-processing, feature selection, and model selection. Many machine learning models developed by experts are designed manually and by trial and error. In other words, even experts need the time and resources to create good predictive machine learning models. The idea of automated machine learning (AutoML) is to automate a machine learning pipeline to release the burden of substantial development costs and manual processes. The algorithms leveraged in these systems have different hyper-parameters. On the other hand, different input datasets have various features. In both cases, the final performance of the model is closely related to the final selected configuration of features and hyper-parameters. That is why they are considered as crucial tasks in the AutoML. The challenges regarding the computationally expensive nature of tuning hyper-parameters and optimally selecting features create significant opportunities for filling the research gaps in the AutoML field. This dissertation explores how to select the features and tune the hyper-parameters of conventional machine learning algorithms efficiently and automatically. To address the challenges in the AutoML area, novel algorithms for hyper-parameter tuning and feature selection are proposed. The hyper-parameter tuning algorithm aims to provide the optimal set of hyper-parameters in three conventional machine learning models (Random Forest, XGBoost and Support Vector Machine) to obtain best scores regarding performance. On the other hand, the feature selection algorithm looks for the optimal subset of features to achieve the highest performance. Afterward, a hybrid framework is designed for both hyper-parameter tuning and feature selection. The proposed framework can discover close to the optimal configuration of features and hyper-parameters. The proposed framework includes the following components: (1) an automatic feature selection component based on artificial bee colony algorithms and machine learning training, and (2) an automatic hyper-parameter tuning component based on artificial bee colony algorithms and machine learning training for faster training and convergence of the learning models. The whole framework has been evaluated using four real-world datasets in different applications. This framework is an attempt to alleviate the challenges of hyper-parameter tuning and feature selection by using efficient algorithms. However, distributed processing, distributed learning, parallel computing, and other big data solutions are not taken into consideration in this framework.

Identifier

FIDC010780

ORCID

0000-0002-7325-1025

Comments

Some chapters are previously published.

Creative Commons License

This work is licensed under a Creative Commons Attribution-Share Alike 2.5 License.

Recommended Citation

Zahedi, Leila, "An Evolutionary Optimization Algorithm for Automated Classical Machine Learning" (2022). FIU Electronic Theses and Dissertations. 5044.
https://digitalcommons.fiu.edu/etd/5044