A Concise Survey on Solving Feature Selection Problems with Metaheuristic Algorithms

With the advancement of industrial digitization, utilizing large datasets for model training to boost performance is a pivotal technical approach for industry progress. However, raw training datasets often contain abundant redundant features, which increase model training's computational cost and impair generalization ability. To tackle this, this study proposes the bionic ABCCOA algorithm, an enhanced version of the bionic Coati Optimization Algorithm (COA), to improve redundant feature elimination in datasets. To address the bionic COA's inadequate global search performance in feature selection (FS) problems, leading to lower classification accuracy, an adaptive search strategy is introduced. This strategy combines individual learning capability and the learnability of disparities, enhancing global exploration. For the imbalance between the exploration and exploitation phases in the bionic COA algorithm when solving FS problems, which often traps it in suboptimal feature subsets, a balancing factor is proposed. By integrating phase control and dynamic adjustability, a good balance between the two phases is achieved, reducing the likelihood of getting stuck in suboptimal subsets. Additionally, to counter the bionic COA's insufficient local exploitation performance in FS problems, increasing classification error rates, a centroid guidance strategy is presented. By combining population centroid guidance and fractional-order historical memory, the algorithm lowers the classification error rate of feature subsets and speeds up convergence. The bionic ABCCOA algorithm was tested on the CEC2020 test functions and engineering problem, achieving an over 90% optimization success rate and faster convergence, confirming its efficiency. Applied to 27 FS problems, it outperformed comparative algorithms in best, average, and worst fitness function values, classification accuracy, feature subset size, and running time, proving it an efficient and robust FS algorithm.

Many evolutionary computation (EC) methods have been used to solve feature selection problems and they perform well on most small-scale feature selection problems. However, as the dimensionality of feature selection problems increases, the solution space increases exponentially. Meanwhile, there are more irrelevant features than relevant features in datasets, which leads to many local optima in the huge solution space. Therefore, the existing EC methods still suffer from the problem of stagnation in local optima on large-scale feature selection problems. Furthermore, large-scale feature selection problems with different datasets may have different properties. Thus, it may be of low performance to solve different large-scale feature selection problems with an existing EC method that has only one candidate solution generation strategy (CSGS). In addition, it is time-consuming to find a suitable EC method and corresponding suitable parameter values for a given large-scale feature selection problem if we want to solve it effectively and efficiently. In this article, we propose a self-adaptive particle swarm optimization (SaPSO) algorithm for feature selection, particularly for large-scale feature selection. First, an encoding scheme for the feature selection problem is employed in the SaPSO. Second, three important issues related to self-adaptive algorithms are investigated. After that, the SaPSO algorithm with a typical self-adaptive mechanism is proposed. The experimental results on 12 datasets show that the solution size obtained by the SaPSO algorithm is smaller than its EC counterparts on all datasets. The SaPSO algorithm performs better than its non-EC and EC counterparts in terms of classification accuracy not only on most training sets but also on most test sets. Furthermore, as the dimensionality of the feature selection problem increases, the advantages of SaPSO become more prominent. This highlights that the SaPSO algorithm is suitable for solving feature selection problems, particularly large-scale feature selection problems.

Whale Optimization Algorithm (WOA) and Grey Wolf Optimizer (GWO) are well-perform metaheuristic algorithms used by various researchers in solving feature selection problems. Yet, the slow convergence speed issue in Whale Optimization Algorithm and Grey Wolf Optimizer could demote the performance of feature selection and classification accuracy. Therefore, to overcome this issue, a modified WOA (mWOA) and modified GWO (mGWO) for wrapper-based feature selection were proposed in this study. The proposed mWOA and mGWO were given a new inversed control parameter which was expected to enable more search area for the search agents in the early phase of the algorithms and resulted in a faster convergence speed. The objective of this comparative study is to investigate and compare the effectiveness of the inversed control parameter in the proposed methods against the original algorithms in terms of the number of selected features and the classification accuracy. The proposed methods were implemented in MATLAB where 12 datasets with different dimensionality from the UCI repository were used. kNN was chosen as the classifier to evaluate the classification accuracy of the selected features. Based on the experimental results, mGWO did not show significant improvements in feature reduction and maintained similar accuracy as the original GWO. On the contrary, mWOA outperformed the original WOA in terms of the two criteria mentioned even on high-dimensional datasets. Evaluating the execution time of the proposed methods, utilizing different classifiers, and hybridizing proposed methods with other metaheuristic algorithms to solve feature selection problems would be future works worth exploring.

Mahalanobis Taguchi system MTS is a classification technique which is used to group objects into dichotomous or binary categories based on numerical predictor variables. MTS uses threshold values based on Mahalanobis distance to group objects into binary classes. An important step in MTS is feature selection whereby a subset of the original set of predictor variables is selected based on maximisation of signal-to-noise S/N ratio. Application of S/N ratio for feature selection in MTS has been criticised for its limitations in handling large number of predictor variables. This research explores the application of artificial bee colony ABC optimisation to select features in MTS. The problem of feature selection is formulated as minimisation of total weighted misclassification and solved as a binary integer programme. The optimal subset of features is obtained via the stochastic search mechanism of binary ABC. The methodology is applied for five case studies. Results are compared with those obtained from two other meta-heuristic techniques namely genetic algorithm and particle swarm optimisation.

Recent growth in data dimensions presents challenges to data mining and machine learning. A high-dimensional dataset consists of several features. Data may include irrelevant or additional features. By removing these redundant and unwanted features, the dimensions of the data can be reduced. The feature selection process eliminates a small set of relevant and important features from a large data set, reducing the size of the dataset. Multiple optimization problems can be solved using metaheuristic algorithms. Recently, the Grasshopper Optimization Algorithm (GOA) has attracted the attention of researchers as a swarm intelligence algorithm based on metaheuristics. An extensive review of papers on GOA-based feature selection algorithms in the years 2018–2023 is presented based on extensive research in the area of feature selection and GOA. A comparison of GOA-based feature selection methods is presented, along with evaluation strategies and simulation environments in this paper. Furthermore, this study summarizes and classifies GOA in several areas. Although many researchers have introduced their novelty in the feature selection problem, many open challenges and enhancements remain. The survey concludes with a discussion about some open research challenges and problems that require further attention.

In mines planning, the long-term production scheduling problem (LTPSP) in open-pit mines is considered as a significant issue. It also specifies the distribution of cash flow during the course of the mine-life. Actually, LTPSP is a large-scale optimisation problem including large data-sets, multiple constraints, and uncertainty in the input factors that, has to be solved in a reasonable time. LTPSP, despite the valuable efforts of researchers, has not yet been well resolved. In this paper, hybrid models have been offered by the Lagrangian relaxation (LR) method with meta-heuristic methods, bat algorithm and particle swarm optimisation for solving the LTPSP due to the deterministic assumption and concerning the grade uncertainty. To bring update the Lagrange multipliers, the meta-heuristic algorithms have been applied. In terms of cumulative net present value, average ore grade, and computational time in a 12-year production period, the consequences achieved from the case studies point out that a solution close to optimisation can be presented by the LR-bat algorithm hybrid strategy in comparison with other methods. The results analysis has shown that the proposed method produces a near-optimal solution with a rational time that can be a good suggestion for utilising in the mining industry.

Technologies like machine learning in the current times, have emerged as capable domains of research in Computer Science. In Machine learning, the system is trained on the basis of the available dataset; the dataset may contain many redundant and irrelevant features which may require more memory for storage and also increases the cost of computation. Selection of best features enhances the accuracy of data classification along with working on smallest amount of features is considered as an optimization problem. Metaheuristic algorithms in current times have been used far and wide unravel various optimization problems. In this context, this study aims to discuss the solution of feature selection problem using metaheuristic algorithms and presents a population based comparison of four metaheuristic algorithms for extracting smallest feature subset with utmost accuracy.

The research and development of metaheuristic methods are critical issues in computer science. In the past decade, metaheuristic algorithms have been used in many engineering applications such as optimization of engineering problems, telecommunications, information security, and image processing. Many metaheuristic algorithms such as particle swarm optimization (PSO), ant colony optimization (ACO), and genetic algorithm (GA) are recently becoming very popular.There are many studies conducted in the literature on the comparison of PSO with other metaheuristic algorithms. In this chapter, various studies carried out between the years of 2010 and 2020 about the comparison of PSO with the other metaheuristic algorithms will be examined. The metaheuristic algorithms to be considered are simulated annealing (SA), genetic algorithm (GA), differential evolution (DE), ant colony optimization (ACO), artificial bee colony (ABC) algorithm, particle swarm optimization (PSO), tabu search (TS), harmony search (HS), firefly algorithm (FF), cuckoo search (CS), bat-inspired algorithm (BA), water wave optimization (WWO), clonal selection algorithm (CLONALG), chemical reaction optimization (CRO), sine cosine algorithm (SCA), glowworm swarm optimization (GSO), and grey wolf optimizer (GWO). This study aims to evaluate and analyze the covered papers according to several criteria such as (a) rates of studies according to publishing years, (b) the metaheuristic algorithms that are compared to PSO, (c) performance evaluation of compared algorithms, (d) the metaheuristic algorithms with their inspirational approaches and their initial proposed studies and years, (e) the field of subjects where the algorithms are applied in the reviewed studies, and (f) used databases in the examined studies.This study is a comprehensive literature review of the comparison of PSO with the most popular metaheuristic algorithms. The intention of this review is to be useful for researchers who want to conduct a survey on this area of the subject as this chapter will cover the essential and helpful analysis of the related research.

Advancements in medical technology have created numerous large datasets including many features. Usually, all captured features are not necessary, and there are redundant and irrelevant features, which reduce the performance of algorithms. To tackle this challenge, many metaheuristic algorithms are used to select effective features. However, most of them are not effective and scalable enough to select effective features from large medical datasets as well as small ones. Therefore, in this paper, a binary moth-flame optimization (B-MFO) is proposed to select effective features from small and large medical datasets. Three categories of B-MFO were developed using S-shaped, V-shaped, and U-shaped transfer functions to convert the canonical MFO from continuous to binary. These categories of B-MFO were evaluated on seven medical datasets and the results were compared with four well-known binary metaheuristic optimization algorithms: BPSO, bGWO, BDA, and BSSA. In addition, the convergence behavior of the B-MFO and comparative algorithms were assessed, and the results were statistically analyzed using the Friedman test. The experimental results demonstrate a superior performance of B-MFO in solving the feature selection problem for different medical datasets compared to other comparative algorithms.

Improved Salp Swarm Algorithm based on opposition based learning and novel local search algorithm for feature selection

Feature selection (FS) is a significant dimensionality reduction technique in machine learning and data mining that is adept at managing high-dimensional data efficiently and enhancing model performance. Metaheuristic algorithms have become one of the most promising solutions in FS owing to their powerful search capabilities as well as their performance. In this paper, the novel improved binary walrus optimizer (WO) algorithm utilizing the golden sine strategy, elite opposition-based learning (EOBL), and population regeneration mechanism (BGEPWO) is proposed for FS. First, the population is initialized using an iterative chaotic map with infinite collapses (ICMIC) chaotic map to improve the diversity. Second, a safe signal is obtained by introducing an adaptive operator to enhance the stability of the WO and optimize the trade-off between exploration and exploitation of the algorithm. Third, BGEPWO innovatively designs a population regeneration mechanism to continuously eliminate hopeless individuals and generate new promising ones, which keeps the population moving toward the optimal solution and accelerates the convergence process. Fourth, EOBL is used to guide the escape behavior of the walrus to expand the search range. Finally, the golden sine strategy is utilized for perturbing the population in the late iteration to improve the algorithm's capacity to evade local optima. The BGEPWO algorithm underwent evaluation on 21 datasets of different sizes and was compared with the BWO algorithm and 10 other representative optimization algorithms. The experimental results demonstrate that BGEPWO outperforms these competing algorithms in terms of fitness value, number of selected features, and F1-score in most datasets. The proposed algorithm achieves higher accuracy, better feature reduction ability, and stronger convergence by increasing population diversity, continuously balancing exploration and exploitation processes and effectively escaping local optimal traps.

Multi-strategy enhanced Marine Predators Algorithm with applications in engineering optimization and feature selection problems

Feature selection is the procedure of extracting the optimal subset of features from an elementary feature set, to reduce the dimensionality of the data. It is an important part of improving the classification accuracy of classification algorithms for big data. Hybrid metaheuristics is one of the most popular methods for dealing with optimization issues. This article proposes a novel feature selection technique called MetaSCA, derived from the standard sine cosine algorithm (SCA). Founded on the SCA, the golden sine section coefficient is added, to diminish the search area for feature selection. In addition, a multi-level adjustment factor strategy is adopted to obtain an equilibrium between exploration and exploitation. The performance of MetaSCA was assessed using the following evaluation indicators: average fitness, worst fitness, optimal fitness, classification accuracy, average proportion of optimal feature subsets, feature selection time, and standard deviation. The performance was measured on the UCI data set and then compared with three algorithms: the sine cosine algorithm (SCA), particle swarm optimization (PSO), and whale optimization algorithm (WOA). It was demonstrated by the simulation data results that the MetaSCA technique had the best accuracy and optimal feature subset in feature selection on the UCI data sets, in most of the cases.

The parameters of the photovoltaic cells and panels are very important to forecast the power generated. There are a lot of methods to extract the parameters using analytical, metaheuristic, and hybrid algorithms. The comparison between the widely used analytical method and some of the best metaheuristic algorithms from the algorithm families is made for datasets from the specialized literature, using the following statistical tests: absolute error, root mean square error, and the coefficient of determination. The equivalent circuit and mathematical model considered is the single diode model. The result comparison shows that the metaheuristic algorithms have the best performance in almost all cases, and only for the genetic algorithm, there are poorer results for one chosen photovoltaic cell. The parameters of the photovoltaic cells and panels and also the current-voltage characteristic for real outdoor weather conditions are forecasted using the parameters calculated with the best method: one for analytical—the five-parameter analytical method—and one for the metaheuristic algorithms—hybrid successive discretization algorithm. Additionally, the genetic algorithm is used. The forecast current-voltage characteristic is compared with the one measured in real sunlight conditions, and the best results are obtained in the case of a hybrid successive discretization algorithm. The maximum power forecast using the calculated parameters with the five-parameter method is the best, and the error in comparison with the measured ones is 0.48%.

Short-term forecasting of streamflow by integrating machine learning methods combined with metaheuristic algorithms