Hybrid binary ant lion optimizer with rough set and approximate entropy reducts for feature selection

In many fields such as data mining, machine learning, pattern recognition and signal processing, data sets containing huge number of features are often involved. Feature selection is an essential data preprocessing technique for such high-dimensional data classification tasks. Traditional dimensionality reduction approach falls into two categories: Feature Extraction (FE) and Feature Selection (FS). Principal component analysis is an unsupervised linear FE method for projecting high-dimensional data into a low-dimensional space with minimum loss of information. It discovers the directions of maximal variances in the data. The Rough set approach to feature selection is used to discover the data dependencies and reduction in the number of attributes contained in a data set using the data alone, requiring no additional information. For selecting discriminative features from principal components, the Rough set theory can be applied jointly with PCA, which guarantees that the selected principal components will be the most adequate for classification. We call this method Rough PCA. The proposed method is successfully applied for choosing the principal features and then applying the Upper and Lower Approximations to find the reduced set of features from a gene expressiondata.

Feature selection based on rough sets and particle swarm optimization

Positive approximation: An accelerator for attribute reduction in rough set theory

With the advancement of industrial digitization, utilizing large datasets for model training to boost performance is a pivotal technical approach for industry progress. However, raw training datasets often contain abundant redundant features, which increase model training's computational cost and impair generalization ability. To tackle this, this study proposes the bionic ABCCOA algorithm, an enhanced version of the bionic Coati Optimization Algorithm (COA), to improve redundant feature elimination in datasets. To address the bionic COA's inadequate global search performance in feature selection (FS) problems, leading to lower classification accuracy, an adaptive search strategy is introduced. This strategy combines individual learning capability and the learnability of disparities, enhancing global exploration. For the imbalance between the exploration and exploitation phases in the bionic COA algorithm when solving FS problems, which often traps it in suboptimal feature subsets, a balancing factor is proposed. By integrating phase control and dynamic adjustability, a good balance between the two phases is achieved, reducing the likelihood of getting stuck in suboptimal subsets. Additionally, to counter the bionic COA's insufficient local exploitation performance in FS problems, increasing classification error rates, a centroid guidance strategy is presented. By combining population centroid guidance and fractional-order historical memory, the algorithm lowers the classification error rate of feature subsets and speeds up convergence. The bionic ABCCOA algorithm was tested on the CEC2020 test functions and engineering problem, achieving an over 90% optimization success rate and faster convergence, confirming its efficiency. Applied to 27 FS problems, it outperformed comparative algorithms in best, average, and worst fitness function values, classification accuracy, feature subset size, and running time, proving it an efficient and robust FS algorithm.

Many evolutionary computation (EC) methods have been used to solve feature selection problems and they perform well on most small-scale feature selection problems. However, as the dimensionality of feature selection problems increases, the solution space increases exponentially. Meanwhile, there are more irrelevant features than relevant features in datasets, which leads to many local optima in the huge solution space. Therefore, the existing EC methods still suffer from the problem of stagnation in local optima on large-scale feature selection problems. Furthermore, large-scale feature selection problems with different datasets may have different properties. Thus, it may be of low performance to solve different large-scale feature selection problems with an existing EC method that has only one candidate solution generation strategy (CSGS). In addition, it is time-consuming to find a suitable EC method and corresponding suitable parameter values for a given large-scale feature selection problem if we want to solve it effectively and efficiently. In this article, we propose a self-adaptive particle swarm optimization (SaPSO) algorithm for feature selection, particularly for large-scale feature selection. First, an encoding scheme for the feature selection problem is employed in the SaPSO. Second, three important issues related to self-adaptive algorithms are investigated. After that, the SaPSO algorithm with a typical self-adaptive mechanism is proposed. The experimental results on 12 datasets show that the solution size obtained by the SaPSO algorithm is smaller than its EC counterparts on all datasets. The SaPSO algorithm performs better than its non-EC and EC counterparts in terms of classification accuracy not only on most training sets but also on most test sets. Furthermore, as the dimensionality of the feature selection problem increases, the advantages of SaPSO become more prominent. This highlights that the SaPSO algorithm is suitable for solving feature selection problems, particularly large-scale feature selection problems.

This paper proposes a local search meta-heuristic free of parameter tuning to solve the attribute reduction problem. Attribute reduction can be defined as the process of finding minimal subset of attributes from an original set with minimum loss of information. Rough set theory has been used for attribute reduction with much success. However, the reduction method inside rough set theory is applicable only to small datasets, since finding all possible reducts is a time consuming process. This motivates many researchers to find alternative approaches to solve the attribute reduction problem. The proposed method, Fuzzy Modified Great Deluge algorithm (Fuzzy-mGD), has one generic parameter which is controlled throughout the search process by using a fuzzy logic controller. Computational experiments confirmed that the Fuzzy-mGD algorithm produces good results, with greater efficiency for attribute reduction, when compared with other meta-heuristic approaches from the literature.

This paper proposes improvements to the binary grey-wolf optimizer (BGWO) to solve the feature selection (FS) problem associated with high data dimensionality, irrelevant, noisy, and redundant data that will then allow machine learning algorithms to attain better classification/clustering accuracy in less training time. We propose three variants of BGWO in addition to the standard variant, applying different transfer functions to tackle the FS problem. Because BGWO generates continuous values and FS needs discrete values, a number of V-shaped, S-shaped, and U-shaped transfer functions were investigated for incorporation with BGWO to convert their continuous values to binary. After investigation, we note that the performance of BGWO is affected by the selection of the transfer function. Then, in the first variant, we look to reduce the local minima problem by integrating an exploration capability to update the position of the grey wolf randomly within the search space with a certain probability; this variant was abbreviated as IBGWO. Consequently, a novel mutation strategy is proposed to select a number of the worst grey wolves in the population which are updated toward the best solution and randomly within the search space based on a certain probability to determine if the update is either toward the best or randomly. The number of the worst grey wolf selected by this strategy is linearly increased with the iteration. Finally, this strategy is combined with IBGWO to produce the second variant of BGWO that was abbreviated as LIBGWO. In the last variant, simulated annealing (SA) was integrated with LIBGWO to search around the best-so-far solution at the end of each iteration in order to identify better solutions. The performance of the proposed variants was validated on 32 datasets taken from the UCI repository and compared with six wrapper feature selection methods. The experiments show the superiority of the proposed improved variants in producing better classification accuracy than the other selected wrapper feature selection algorithms.

In this paper we study a model to feature selection based on Ant Colony Optimization and Rough Set Theory. The algorithm looks for reducts by using ACO as search method and RST offers the heuristic function to measure the quality of one feature subset. Major results of using this approach are shown and others are referenced. Recently, runtime analyses of Ant Colony Optimization algorithms have been studied also. However the efforts are limited to specific classes of problems or simplified algorithm’s versions, in particular studying a specific part of the algorithms like the pheromone influence. From another point of view, this paper presents results of applying an improved ACO implementation which focuses on decreasing the number of heuristic function evaluations needed.

Improved Salp Swarm Algorithm based on opposition based learning and novel local search algorithm for feature selection

In many data analysis tasks, one is often confronted with very high dimensional data. Feature selection techniques are designed to find the relevant feature subset of the original features which can facilitate clustering, classification and retrieval. In this paper, we consider the feature selection problem in unsupervised learning scenario, which is particularly difficult due to the absence of class labels that would guide the search for relevant information. The feature selection problem is essentially a combinatorial optimization problem which is computationally expensive. Traditional unsupervised feature selection methods address this issue by selecting the top ranked features based on certain scores computed independently for each feature. These approaches neglect the possible correlation between different features and thus can not produce an optimal feature subset. Inspired from the recent developments on manifold learning and L1-regularized models for subset selection, we propose in this paper a new approach, called Multi-Cluster Feature Selection (MCFS), for unsupervised feature selection. Specifically, we select those features such that the multi-cluster structure of the data can be best preserved. The corresponding optimization problem can be efficiently solved since it only involves a sparse eigen-problem and a L1-regularized least squares problem. Extensive experimental results over various real-life data sets have demonstrated the superiority of the proposed algorithm.

Feature selection is an important research issue in classification. As an effective global optimization technique, Particle Swarm Optimization (PSO) algorithm has been widely employed to solve feature selection problems. Population initialization is an important stage in Evolutionary Computation (EC) techniques. Therefore, in EC research field, many works focus on population initialization and it has been verified that suitable population initialization methods can significantly improve the performance of EC techniques. The efficient of PSO significantly deteriorates when being used to solve the large-scale feature selection problems. At present, only a few works have been done to enhance the performance of PSO for feature selection problems through population initialization. Meanwhile, there are many excellent traditional filter feature selection methods. Their characteristic is quick in speed but low in effect. However, they can provide a lot of useful heuristic information. Therefore, this paper focuses on improving the performance of PSO for feature selection problems by designing filter-based population initialization methods. In the proposed method, the filters are firstly used to evaluate features. Based on the obtained heuristic information, the initialization method combining the mixed initialization and the threshold selection is designed. The experiments are carried out on several datasets and the proposed initialization method is compared to some related initialization methods. Besides, the K-Nearest Neighbour (KNN) and other compatible fitness learners are used for performance assessment for feature selection. The results show that the proposed method using KNN classifier is promising to improve the performance of PSO for feature selection problems.

Most classification studies are done by using all the objects data. It is expected to classify objects by using some subsets data in the total data. A rough set based reduct is a minimal subset of features, which has almost the same discernible power as the entire conditional features. Here, we propose a greedy algorithm to compute a set of rough set reducts which is followed by the k-nearest neighbor to classify documents. To improve the classification performance, reducts-kNN with confidence was developed. These proposed rough set reduct based methods are compared with the classification by AdaBoost and SVM(Support Vector Machine) methods. Experiments have been conducted on some benchmark datasets from the Reuters 21578 data set.

Feature selection is an important data preprocessing technique in classification problems. This paper focuses on a new feature selection problem, in which sampling data of different features have different reliability degree. First, the problem is modeled as a multi-objective optimization. There two objectives should be optimized simultaneously: reliability and classifying accuracy of feature subset. Then, a multi-objective feature selection method based on particle swarm optimization, called JMOPSO, is proposed by incorporating several effective operators. Finally, experimental results suggest that the proposed JMOPSO is a highly competitive feature selection method for solving the feature selection problem with unreliable data.

Feature Selection methods in Data Mining and Data Analysis problems aim at selecting a subset of the variables, or features, that describe the data in order to obtain a more essential and compact representation of the available information. The selected subset has to be small in size and must retain the information that is most useful for the specific application. The role of Feature Selection is particularly important when computationally expensive Data Mining tools are used, or when the data collection process is difficult or costly. Feature Selection problems are typically solved in the literature using search techniques, where the evaluation of a specific subset is accomplished by a proper function (filter methods) or directly by the performance of a Data Mining tool (wrapper methods). In this work we show how the Feature Selection problem can be formulated as a subgraph selection problem derived from the lightest k-subgraph problem, and solved as an Integer Program. The proposed formulation is very flexible, as additional conditions on the solution can be added in the formulation. Although optimal solutions for such problems are difficult to find in the worst case, a large number of test instances have been solved efficiently by commercial tools. Finally, an application to a database on urban mobility is presented, where the proposed method is integrated in the Data Mining tool named Lsquare and is compared with other approaches.

The problem of feature selection has been an area of considerable research in machine learning. Feature selection is known to be particularly difficult in unsupervised learning because different subgroups of features can yield useful insights into the same dataset. In other words, many theoretically-right answers may exist for the same problem. Furthermore, designing algorithms for unsupervised feature selection is technically harder than designing algorithms for supervised feature selection because unsupervised feature selection algorithms cannot be guided by class labels. As a result, previous work attempts to discover intrinsic structures of data with heavy computation such as matrix decomposition, and require significant time to find even a single solution. This paper proposes a novel algorithm, named Explainability-based Unsupervised Feature Value Selection (EUFVS), which enables a paradigm shift in feature selection, and solves all of these problems. EUFVS requires only a few tens of milliseconds for datasets with thousands of features and instances, allowing the generation of a large number of possible solutions and select the solution with the best fit. Another important advantage of EUFVS is that it selects feature values instead of features, which can better explain phenomena in data than features. EUFVS enables a paradigm shift in feature selection. This paper explains its theoretical advantage, and also shows its applications in real experiments. In our experiments with labeled datasets, EUFVS found feature value sets that explain labels, and also detected useful relationships between feature value sets not detectable from given class labels.KeywordsFeature selectionUnsupervised learningClustering