AN OPTIMAL FEATURE SUBSET SELECTION METHOD BASED ON DISTANCE DISCRIMINANT AND DISTRIBUTION OVERLAPPING

Computer-aided diagnosis has the potential of increasing diagnostic accuracy by providing a second reading to radiologists. In many computerized schemes, numerous features can be extracted to describe suspect image regions. A subset of these features is then employed in a data classifier to determine whether the suspect region is abnormal or normal. Different subsets of features will, in general, result in different classification performances. A feature selection method is often used to determine an "optimal" subset of features to use with a particular classifier. A classifier performance measure (such as the area under the receiver operating characteristic curve) must be incorporated into this feature selection process. With limited datasets, however, there is a distribution in the classifier performance measure for a given classifier and subset of features. In this paper, we investigate the variation in the selected subset of "optimal" features as compared with the true optimal subset of features caused by this distribution of classifier performance. We consider examples in which the probability that the optimal subset of features is selected can be analytically computed. We show the dependence of this probability on the dataset sample size, the total number of features from which to select, the number of features selected, and the performance of the true optimal subset. Once a subset of features has been selected, the parameters of the data classifier must be determined. We show that, with limited datasets and/or a large number of features from which to choose, bias is introduced if the classifier parameters are determined using the same data that were employed to select the "optimal" subset of features.

Feature selection is the process of choosing, or removing, features to obtain the most informative feature subset of minimal size. Such subsets are used to improve performance of machine learning algorithms and enable human understanding of the results. Approaches to feature selection in literature exploit several optimization algorithms. Multi-objective methods also have been proposed, minimizing at the same time the number of features and the error. While most approaches assess error resorting to the average of a stochastic K-fold cross-validation, comparing averages might be misleading. In this paper, we show how feature subsets with different average error might in fact be non-separable when compared using a statistical test. Following this idea, clusters of non-separable optimal feature subsets are identified. The performance in feature selection can thus be evaluated by verifying how many of these optimal feature subsets an algorithm is able to identify. We thus propose a multi-objective optimization approach to feature selection, EvoFS, with the objectives to i. minimize feature subset size, ii. minimize test error on a 10-fold cross-validation using a specific classifier, iii. maximize the analysis of variance value of the lowest-performing feature in the set. Experiments on classification datasets whose feature subsets can be exhaustively evaluated show that our approach is able to always find the best feature subsets. Further experiments on a high-dimensional classification dataset, that cannot be exhaustively analyzed, show that our approach is able to find more optimal feature subsets than state-of-the-art feature selection algorithms.

Hyperspectral images (HSIs) generally contain a large amount of spectral bands (features), and the redundant information in them will cause the Hughes phenomenon in the classification process. And feature extraction and feature selection are the two main existing methods to effectively reduce the redundancy of spectral information in the field of HSIs classification. Compared with feature extraction methods, feature selection methods can preserve most of the features of the original HSIs data without losing their valuable details. However, most existing feature selection methods based on single scene (domain) perform poorly in some scenes (domains) with insufficient labeled samples. Therefore, how to adopt an efficient feature selection method to select the optimal feature subsets of source scene and target scene and use the sample information of source scene to assist in the classification of target scene so as to improve the classification accuracy of images in the target scene as much as possible is still very challenging. In order to solve the above problem, this paper proposes a new cross-scene algorithm: Improved Ant Colony Optimization Algorithm-Based Cross-Scene Feature Selection Algorithm (IMACO-CSFS). In order to obtain more accurate feature subsets of the two scenes, IMACO-CSFS proposes a priority sorting-based ant colony strategy to make the subsequent search process focus on the global optimal solution (optimal feature subset) found in the previous iteration. In addition, in order to further accelerate the convergence speed of the global optimal solution, an ant colony strategy based on elite ants is proposed in IMACO-CSFS to more efficiently obtain the optimal feature subsets of the two scenes for training the classifier. Furthermore, this paper simultaneously considers overall classification accuracies of the optimal feature subsets for both scenes and dynamically adjusts their scale to ensure the consistency of the selected features between the two scenes, attenuating the effect of spectral shift and achieving the higher image classification accuracy in the target scene. Experimental results on three cross-scene HSI data pairs demonstrate that IMACO-CSFS is superior in cross-scene feature selection and cross-scene HSIs classification.

Feature selection has become a de facto tool for analyzing high-dimensional data, especially in bioinformatics. It is effective in improving learning algorithms’ scalability and facilitating feature generalization or interpretability by removing noise and redundancy. Our focus is placed on the paradigm of supervised feature selection, which aims to find an optimal feature subset to best predict the target. We propose a nonlinear approach for finding a feature subset that achieves the highest inter-class separability in terms of the kernel Discriminant Information (KDI) measure. Theoretically, we prove the existence of good prediction hypotheses for feature subsets with high KDI value. We also establish the equivalency between maximizing the KDI statistic and minimizing a functional dependency measure of label variable on data. Moreover, we asymptotically prove the concentration property of the optimal feature subset found by maximizing the KDI measure. Practically, we provide an efficient gradient optimization algorithm for solving the KDI feature selection problem. We evaluate the proposed method based on 19 benchmark datasets in various domains, and demonstrates a noticeable improvement against state-of-the-art baselines on the majority of classification and regression tasks. Notably, our method is robust to the choice of hyper-parameters, works well with various downstream classifiers, has competitive computational complexity among the kernel based methods considered, and scales well the large-scale object recognition dataset, with generalization enhancement on CIFAR.

In order to obtain the higher classification accuracy in specific categories for the different feature subset, a hierarchical classification algorithm based on Feature Selection is proposed, and is used for synthetic aperture radar (SAR) image classification, and feature selection is achieved by Genetic algorithm. The algorithm has two main characteristics: one is hierarchical classification which consists of many two-class classifier, and the two-class classifier is trained by the optimal feature subset which is selected according to different categories; the second is the classifier of support vector machine SVM (Support Vector Machine); the two is Genetic algorithm which can search out the optimal feature subset and parameters of support vector machine that is most suitable for the category, by unified coding the feature set and the parameters of SVM to constitute the Chromosome. The experiment on the first polarimetric SAR data show that the algorithm can obtain higher classification accuracy rate.

In the field of hand gesture recognition based on electromyography, the selection of features is still a matter of research because the accuracy of feature evaluation techniques depends on the distribution underling a feature space. Currently, there are a lot of kinds of features, so a selection method is required in order to find an optimal feature subset. The selection of features can be done based on either classifier methods or statistical measurement indexes. On one hand, the use of classifier methods takes considerable time, consequently, they are computationally expensive. On the other hand, the use of a statistical index allows to select features based on properties, and they are less expensive in terms of time and computational costs. In this context, this study analyzed different statistical indexes to discover optimal feature subsets and predicts which one may have the best performance in terms of classification and recognition accuracy. For this purpose, four indexes were analyzed: the sum of RES (separation and compactness index), the sum of Fuzzy Entropies, and both of these indexes divided by the mutual information of feature subset. These index values were calculated for 10 different feature subsets obtained from nineteen time-domain features which have been investigated by many researchers for extracting useful information from an electromyography signal. The feature subsets were implemented in four classifiers (Artificial Neural Network, k-nearest Neighbors Algorithm, Support Vector Machine, and decision trees) to find the best statistical index which tries to quantify the suitableness of the feature space. Each tested index in this work was analyzed in terms of classification and recognition accuracy for each machine learning algorithm.KeywordsHand gesture recognitionElectromyographyFeature evaluationFuzzy entropyMutual informationMyo armband sensor

Classification of omics data suffers from the high error rate due to their high dimensional and small sample size characteristics. To overcome the problem, this paper proposes an ensemble feature selection for omics data classification based on constrained niching binary particle swarm optimization (PSO). Particularly, optimal feature subsets in terms of best classification accuracy are identified by the binary PSO. The proposed method introduces constraint on the particle encoding to constrain the number of selected features, and niching technique from multimodal optimization is imposed to enable the algorithm to obtain multiple diverse feature subsets in a single run. Afterward, multiple base classifiers built on the obtained feature subsets are combined into a stronger classifier which is applied to classify the omics data. Experimental results on real-world omics datasets demonstrate that the proposed feature selection method can stably select compact feature subsets and obtain promising classification performance.

Solar irradiation is influenced by many meteorological features, which results in a complex structure meaning its prediction has low efficiency and accuracy. The existing prediction methods are focused on analyzing the correlation between features and irradiation to reduce model complexity but they do not account for redundant analysis in feature subset. In order to reduce the information redundancy in the feature set and improve prediction accuracy, a novel feature selection method for short-term irradiation prediction based on Conditional Mutual Information (CMI) and Gaussian Process Regression (GPR) is proposed. Firstly, the CMI values of different features are calculated to evaluate correlation and redundant information between features in the feature subsets. Secondly, GPR with a stable prediction performance and adaptively determined hyper parameters is used as the predictor. The optimal feature subset and the GPR covariance function can be selected using Sequential Forward Selection (SFS). Finally, an optimal predictor is determined by the minimum prediction error and the prediction of solar irradiation is carried out by the determined predictor. The experimental results show that CMI-GPRAEK has the highest prediction accuracy with the optimal feature set has low dimension, which is 4.33% lower in MAPE than the predictor without feature selection, although both of them have an optimal kernel function. The CMI-GPRAEK is less complicated for the predictor and there is less redundancy between features in the model with the dimension of the optimal feature set is only 14.

A new hybrid filter/wrapper algorithm for feature selection in classification

Feature selection (FS) has been studied as a multi-objective problem in recent years to improve the classification effect. In this paper, to expand the ability of Bacterial Foraging Optimization (BFO) in feature selection problems for classification, a Multi-objective Adapting Chemotaxis Bacterial Foraging Optimization (abbreviated as MOACBFO) is proposed. In MOACBFO, a structural variation strategic model is proposed to reduce the computational complexity for multi-objective problems and applied with a dynamically updated external matrix to record the performance of bacteria on two objectives. In addition, an adaptive chemotaxis step mechanism is designed to help the bacteria jump out of the local optimality. To further enhance the diversity of bacteria searching capability, a feature subset updating strategy is developed. The optimal feature subset and fitness value are stored in the external matrix continuously by comparing them with the historical value records. The performance of the proposed algorithm is demonstrated by comparing it with four other advanced swarm intelligent algorithms on 11 high-dimensional microarray datasets. The results indicate that the MOACBFO performs better in achieving a lower classification error rate using a small number of features.

Large-scale feature selection is one of the most important fields in the big data domain that can solve real data problems, such as bioinformatics, where it is necessary to process huge amount of data. The efficiency of existing feature selection algorithms significantly downgrades, if not totally inapplicable, when data size exceeds hundreds of gigabytes, because most feature selection algorithms are designed for centralized computing architecture. For that, distributed computing techniques, such as MapReduce can be applied to handle very large data. Our approach is to scale the existing method for feature selection, Kmeans clustering and Signal to Noise Ratio (SNR) combined with optimization technique as Binary Particle Swarm Optimization (BPSO). The proposed method is divided into two stages. In the first stage, we have used parallel Kmeans on MapReduce for clustering features, and then we have applied iterative MapReduce that implement parallel SNR ranking for each cluster. After, we have selected the top ranked feature from each cluster. The top scored features from each cluster are gathered and a new feature subset is generated. In the second stage, the new feature subset is used as input to the proposed BPSO based on MapReduce which provides an optimized feature subset. The proposed method is implemented in a distributed environment, and its efficiency is illustrated through analyzing practical problems such as biomarker discovery.

Gene expression profile data have high-dimensionality with a small number of samples. These data characteristics lead to a long training time and low performance in predictive model construction. To address this issue, the paper proposes a feature selection algorithm using non-dominant feature-guide search. The algorithm adopts a filtering framework based on feature sorting and search strategy to overcome the problems of long training time and poor performance. First, the feature pre-selection is completed according to the calculated feature category correlation. Second, a multi-objective optimization feature selection model is constructed. Non-dominant features are defined according to the Pareto dominance theory. Combined with the bidirectional search strategy, the Pareto dominance features under the current category maximum relevance feature are removed one by one. Finally, the optimal feature subset with maximum correlation and minimum redundancy is obtained. Experimental results on six gene expression data sets show that the algorithm is much better than Fisher score, maximum information coefficient, composition of feature relevancy, mini-batch K-means normalized mutual information feature inclusion, and max-Relevance and Min-Redundancy algorithms. Compared to feature selection method based on maximum information coefficient and approximate Markov blanket, the algorithm not only has high computational efficiency but also can obtain better classification capabilities in a smaller dimension.

Feature selection is the procedure of extracting the optimal subset of features from an elementary feature set, to reduce the dimensionality of the data. It is an important part of improving the classification accuracy of classification algorithms for big data. Hybrid metaheuristics is one of the most popular methods for dealing with optimization issues. This article proposes a novel feature selection technique called MetaSCA, derived from the standard sine cosine algorithm (SCA). Founded on the SCA, the golden sine section coefficient is added, to diminish the search area for feature selection. In addition, a multi-level adjustment factor strategy is adopted to obtain an equilibrium between exploration and exploitation. The performance of MetaSCA was assessed using the following evaluation indicators: average fitness, worst fitness, optimal fitness, classification accuracy, average proportion of optimal feature subsets, feature selection time, and standard deviation. The performance was measured on the UCI data set and then compared with three algorithms: the sine cosine algorithm (SCA), particle swarm optimization (PSO), and whale optimization algorithm (WOA). It was demonstrated by the simulation data results that the MetaSCA technique had the best accuracy and optimal feature subset in feature selection on the UCI data sets, in most of the cases.

In recent years, detecting credit card fraud transactions has been a difficult task due to the high dimensions and imbalanced datasets. Selecting a subset of important features from a high-dimensional dataset has proven to be the most prominent approach for solving high-dimensional dataset issues, and the selection of features is critical for improving classification performance, such as the fraud transaction identification process. To contribute to the field, this paper proposes a novel feature selection (FS) approach based on a metaheuristic algorithm called Rock Hyrax Swarm Optimization Feature Selection (RHSOFS), inspired by the actions of rock hyrax swarms in nature, and implements supervised machine learning techniques to improve credit card fraud transaction identification approaches. This approach is used to select a subset of optimal relevant features from a high-dimensional dataset. In a comparative efficiency analysis, RHSOFS is compared with Differential Evolutionary Feature Selection (DEFS), Genetic Algorithm Feature Selection (GAFS), Particle Swarm Optimization Feature Selection (PSOFS), and Ant Colony Optimization Feature Selection (ACOFS) in a comparative efficiency analysis. The proposed RHSOFS outperforms existing approaches, such as DEFS, GAFS, PSOFS, and ACOFS, according to the experimental results. Various statistical tests have been used to validate the statistical significance of the proposed model.

This paper proposes an effective feature selection method based on monarch butterfly optimization and Fisher criterion. Fisher criterion is applied to evaluate the feature subsets, based on which the optimal feature subsets are searched by using monarch butterfly optimization algorithm. To combine these two components, a method is developed to binarize continuous solution vectors for deciding the feature selection. We conduct experiments on widely used UCI (University of California, Irvine) classification datasets to study the design of our algorithm and compare it with other state-of-the-art counterparts. The experimental results show that the proposed method is reasonable and effective, which achieves the best result of feature selection among the compared methods and has satisfactory efficiency.