A New Feature Selection Method based on Monarch Butterfly Optimization and Fisher Criterion

Computer-aided diagnosis has the potential of increasing diagnostic accuracy by providing a second reading to radiologists. In many computerized schemes, numerous features can be extracted to describe suspect image regions. A subset of these features is then employed in a data classifier to determine whether the suspect region is abnormal or normal. Different subsets of features will, in general, result in different classification performances. A feature selection method is often used to determine an "optimal" subset of features to use with a particular classifier. A classifier performance measure (such as the area under the receiver operating characteristic curve) must be incorporated into this feature selection process. With limited datasets, however, there is a distribution in the classifier performance measure for a given classifier and subset of features. In this paper, we investigate the variation in the selected subset of "optimal" features as compared with the true optimal subset of features caused by this distribution of classifier performance. We consider examples in which the probability that the optimal subset of features is selected can be analytically computed. We show the dependence of this probability on the dataset sample size, the total number of features from which to select, the number of features selected, and the performance of the true optimal subset. Once a subset of features has been selected, the parameters of the data classifier must be determined. We show that, with limited datasets and/or a large number of features from which to choose, bias is introduced if the classifier parameters are determined using the same data that were employed to select the "optimal" subset of features.

Feature selection is the process of choosing, or removing, features to obtain the most informative feature subset of minimal size. Such subsets are used to improve performance of machine learning algorithms and enable human understanding of the results. Approaches to feature selection in literature exploit several optimization algorithms. Multi-objective methods also have been proposed, minimizing at the same time the number of features and the error. While most approaches assess error resorting to the average of a stochastic K-fold cross-validation, comparing averages might be misleading. In this paper, we show how feature subsets with different average error might in fact be non-separable when compared using a statistical test. Following this idea, clusters of non-separable optimal feature subsets are identified. The performance in feature selection can thus be evaluated by verifying how many of these optimal feature subsets an algorithm is able to identify. We thus propose a multi-objective optimization approach to feature selection, EvoFS, with the objectives to i. minimize feature subset size, ii. minimize test error on a 10-fold cross-validation using a specific classifier, iii. maximize the analysis of variance value of the lowest-performing feature in the set. Experiments on classification datasets whose feature subsets can be exhaustively evaluated show that our approach is able to always find the best feature subsets. Further experiments on a high-dimensional classification dataset, that cannot be exhaustively analyzed, show that our approach is able to find more optimal feature subsets than state-of-the-art feature selection algorithms.

After studying the behavior of monarch butterflies in nature, Wang et al. proposed a new promising swarm intelligence algorithm, called monarch butterfly optimization (MBO), for addressing unconstrained optimization tasks. In the basic MBO algorithm, the fixed butterfly adjusting rate is used to carry out the butterfly adjusting operator. In this paper, the self-adaptive strategy is introduced to adjust the butterfly adjusting rate. In addition, the crossover operator that is generally used in evolutionary algorithms (EAs) is used to further improve the quality of butterfly individuals. The two optimization strategies, self-adaptive and crossover operator, are combined, and then self-adaptive crossover operator is proposed. After incorporating the above strategies into the basic MBO algorithm, a new version of MBO algorithm, called Self-adaptive Monarch Butterfly Optimization (SaMBO), is put forward. Also, few studies of constrained optimization has been done for MBO research. In this paper, in order to verify the performance of our proposed SaMBO algorithm, the proposed SaMBO algorithm is further benchmarked by 21 CEC 2017 constrained optimization problems. The experimental results indicate that the proposed SaMBO algorithm outperforms the basic MBO and other five state-of-the-art metaheuristic algorithms.

Hyperspectral images (HSIs) generally contain a large amount of spectral bands (features), and the redundant information in them will cause the Hughes phenomenon in the classification process. And feature extraction and feature selection are the two main existing methods to effectively reduce the redundancy of spectral information in the field of HSIs classification. Compared with feature extraction methods, feature selection methods can preserve most of the features of the original HSIs data without losing their valuable details. However, most existing feature selection methods based on single scene (domain) perform poorly in some scenes (domains) with insufficient labeled samples. Therefore, how to adopt an efficient feature selection method to select the optimal feature subsets of source scene and target scene and use the sample information of source scene to assist in the classification of target scene so as to improve the classification accuracy of images in the target scene as much as possible is still very challenging. In order to solve the above problem, this paper proposes a new cross-scene algorithm: Improved Ant Colony Optimization Algorithm-Based Cross-Scene Feature Selection Algorithm (IMACO-CSFS). In order to obtain more accurate feature subsets of the two scenes, IMACO-CSFS proposes a priority sorting-based ant colony strategy to make the subsequent search process focus on the global optimal solution (optimal feature subset) found in the previous iteration. In addition, in order to further accelerate the convergence speed of the global optimal solution, an ant colony strategy based on elite ants is proposed in IMACO-CSFS to more efficiently obtain the optimal feature subsets of the two scenes for training the classifier. Furthermore, this paper simultaneously considers overall classification accuracies of the optimal feature subsets for both scenes and dynamically adjusts their scale to ensure the consistency of the selected features between the two scenes, attenuating the effect of spectral shift and achieving the higher image classification accuracy in the target scene. Experimental results on three cross-scene HSI data pairs demonstrate that IMACO-CSFS is superior in cross-scene feature selection and cross-scene HSIs classification.

Feature selection is one of the most significant procedures in machine learning algorithms. It is particularly to improve the performance and prediction accuracy for complex data classification. This paper discusses a hybrid feature selection technique with the decision tree-based classification algorithm. The feature selected using information gain (IG) is combined with the features selected from ReliefF which generates the resultant feature subset. Then the resultant feature subset is in turn combined with a correlation-based feature selection (CFS) method to generate the aggregated feature subset. To perform classification accuracy on the aggregated feature subset, different decision trees-based classification algorithm such as C4.5, decision stumps, naive Bayes tree, and random forest with ten-fold cross-validation have been deployed. To check the prediction accuracy of the proposed work eight different multilevel University of California, Irvine (UCI) machine learning datasets have been used with minimum to maximum numbers of features. The main objective of the hybrid feature selection is to improve the classification accuracy, prediction and to reduce the execution time using standard datasets.

A hybrid algorithm using ant and bee colony optimization for feature selection and classification (AC-ABC Hybrid)

Monarch butterfly optimization is inspired by the migration of monarch butterflies in nature: monarch butterflies in eastern North America migrate southward from northern United States and southern Canada to Mexico every summer/autumn. It is a new swarm intelligence metaheuristic algorithm, which is called monarch butterfly optimization (MBO). In MBO, the monarch butterfly population is ideally divided into two populations, one is located in southern Canada and northern United States (land 1), and the other is located in Mexico (land 2). We update the individuals in the population through migration and butterfly shape adjustment. To maintain the diversity of the population and minimize the fitness value, the total number of butterflies generated by the operation should be equal to the original population. In the experiment, MBO is compared with five metaheuristic algorithms through 38 test functions, especially applied to solve a large-scale 0-1 knapsack (KP) problem. Three groups of 15 large-scale 0-1 KP cases from 800 to 2000 dimensions are used to test. It is proved that MBO has strong advantages in solving global numerical optimization and engineering problems, which provides a reference for further discussion of MBO in the future.

As a significant subset of the family of discrete optimization problems, the 0-1 knapsack problem (0-1 KP) has received considerable attention among the relevant researchers. The monarch butterfly optimization (MBO) is a recent metaheuristic algorithm inspired by the migration behavior of monarch butterflies. The original MBO is proposed to solve continuous optimization problems. This paper presents a novel monarch butterfly optimization with a global position updating operator (GMBO), which can address 0-1 KP known as an NP-complete problem. The global position updating operator is incorporated to help all the monarch butterflies rapidly move towards the global best position. Moreover, a dichotomy encoding scheme is adopted to represent monarch butterflies for solving 0-1 KP. In addition, a specific two-stage repair operator is used to repair the infeasible solutions and further optimize the feasible solutions. Finally, Orthogonal Design (OD) is employed in order to find the most suitable parameters. Two sets of low-dimensional 0-1 KP instances and three kinds of 15 high-dimensional 0-1 KP instances are used to verify the ability of the proposed GMBO. An extensive comparative study of GMBO with five classical and two state-of-the-art algorithms is carried out. The experimental results clearly indicate that GMBO can achieve better solutions on almost all the 0-1 KP instances and significantly outperforms the rest.

In order to obtain the higher classification accuracy in specific categories for the different feature subset, a hierarchical classification algorithm based on Feature Selection is proposed, and is used for synthetic aperture radar (SAR) image classification, and feature selection is achieved by Genetic algorithm. The algorithm has two main characteristics: one is hierarchical classification which consists of many two-class classifier, and the two-class classifier is trained by the optimal feature subset which is selected according to different categories; the second is the classifier of support vector machine SVM (Support Vector Machine); the two is Genetic algorithm which can search out the optimal feature subset and parameters of support vector machine that is most suitable for the category, by unified coding the feature set and the parameters of SVM to constitute the Chromosome. The experiment on the first polarimetric SAR data show that the algorithm can obtain higher classification accuracy rate.

A new hybrid filter/wrapper algorithm for feature selection in classification

Feature selection has become a de facto tool for analyzing high-dimensional data, especially in bioinformatics. It is effective in improving learning algorithms’ scalability and facilitating feature generalization or interpretability by removing noise and redundancy. Our focus is placed on the paradigm of supervised feature selection, which aims to find an optimal feature subset to best predict the target. We propose a nonlinear approach for finding a feature subset that achieves the highest inter-class separability in terms of the kernel Discriminant Information (KDI) measure. Theoretically, we prove the existence of good prediction hypotheses for feature subsets with high KDI value. We also establish the equivalency between maximizing the KDI statistic and minimizing a functional dependency measure of label variable on data. Moreover, we asymptotically prove the concentration property of the optimal feature subset found by maximizing the KDI measure. Practically, we provide an efficient gradient optimization algorithm for solving the KDI feature selection problem. We evaluate the proposed method based on 19 benchmark datasets in various domains, and demonstrates a noticeable improvement against state-of-the-art baselines on the majority of classification and regression tasks. Notably, our method is robust to the choice of hyper-parameters, works well with various downstream classifiers, has competitive computational complexity among the kernel based methods considered, and scales well the large-scale object recognition dataset, with generalization enhancement on CIFAR.

In nature, the eastern North American monarch population is known for its southward migration during the late summer/autumn from the northern USA and southern Canada to Mexico, covering thousands of miles. By simplifying and idealizing the migration of monarch butterflies, a new kind of nature-inspired metaheuristic algorithm, called monarch butterfly optimization (MBO), a first of its kind, is proposed in this paper. In MBO, all the monarch butterfly individuals are located in two distinct lands, viz. southern Canada and the northern USA (Land 1) and Mexico (Land 2). Accordingly, the positions of the monarch butterflies are updated in two ways. Firstly, the offsprings are generated (position updating) by migration operator, which can be adjusted by the migration ratio. It is followed by tuning the positions for other butterflies by means of butterfly adjusting operator. In order to keep the population unchanged and minimize fitness evaluations, the sum of the newly generated butterflies in these two ways remains equal to the original population. In order to demonstrate the superior performance of the MBO algorithm, a comparative study with five other metaheuristic algorithms through thirty-eight benchmark problems is carried out. The results clearly exhibit the capability of the MBO method toward finding the enhanced function values on most of the benchmark problems with respect to the other five algorithms. Note that the source codes of the proposed MBO algorithm are publicly available at GitHub ( https://github.com/ggw0122/Monarch-Butterfly-Optimization , C++/MATLAB) and MATLAB Central ( http://www.mathworks.com/matlabcentral/fileexchange/50828-monarch-butterfly-optimization , MATLAB).

Picking a subset of possible features is a vital step in the data-mining procedure. The decisive goal of feature-selection is to determine the optimal number of superior characteristics to make best use of the presentation of the learning algorithm. However, this problem becomes increasingly challenging to resolve as the number of features in a data set increase. Therefore, to identify the optimal feature combinations, contemporary optimization techniques are used. Numerous optimization problems have been successfully resolved using the innovative metaheuristic known as the marine predators algorithm (MPA). Support vector machines (SVMs) are a crucial technique that is expertly applied to classification issues. In this work, the issue of feature picking in large dimensional data sets is solved by adjusting the MPA using the SVM as a classifier. In order to address the problem of feature selection in large dimensional data sets, the current study suggests MPA + SVM. Ten high-dimensional data sets got from the Arizona State University (ASU) source were employed to prove the usefulness of the planned method; the outcomes are likened with those of the additional six cutting-edge picking features algorithms. We compared the following algorithms: atom search optimization (ASO), satin bowerbird optimizer (SBO), emperor penguin optimizer (EPO), equilibrium optimizer (EO), monarch butterfly optimization (MBO), and sine cosine algorithm (SCA). The consequences confirm that the planned MPA+SVM approach outperformed several metaheuristic algorithms and presented a remarkable capability to pick the utmost weighty and optimum features. MPA + SVM yields the lowermost averaged error rates, minimal classification standard deviation (STD) values, and FS rates across all data sets.

In the field of hand gesture recognition based on electromyography, the selection of features is still a matter of research because the accuracy of feature evaluation techniques depends on the distribution underling a feature space. Currently, there are a lot of kinds of features, so a selection method is required in order to find an optimal feature subset. The selection of features can be done based on either classifier methods or statistical measurement indexes. On one hand, the use of classifier methods takes considerable time, consequently, they are computationally expensive. On the other hand, the use of a statistical index allows to select features based on properties, and they are less expensive in terms of time and computational costs. In this context, this study analyzed different statistical indexes to discover optimal feature subsets and predicts which one may have the best performance in terms of classification and recognition accuracy. For this purpose, four indexes were analyzed: the sum of RES (separation and compactness index), the sum of Fuzzy Entropies, and both of these indexes divided by the mutual information of feature subset. These index values were calculated for 10 different feature subsets obtained from nineteen time-domain features which have been investigated by many researchers for extracting useful information from an electromyography signal. The feature subsets were implemented in four classifiers (Artificial Neural Network, k-nearest Neighbors Algorithm, Support Vector Machine, and decision trees) to find the best statistical index which tries to quantify the suitableness of the feature space. Each tested index in this work was analyzed in terms of classification and recognition accuracy for each machine learning algorithm.KeywordsHand gesture recognitionElectromyographyFeature evaluationFuzzy entropyMutual informationMyo armband sensor

Classification of omics data suffers from the high error rate due to their high dimensional and small sample size characteristics. To overcome the problem, this paper proposes an ensemble feature selection for omics data classification based on constrained niching binary particle swarm optimization (PSO). Particularly, optimal feature subsets in terms of best classification accuracy are identified by the binary PSO. The proposed method introduces constraint on the particle encoding to constrain the number of selected features, and niching technique from multimodal optimization is imposed to enable the algorithm to obtain multiple diverse feature subsets in a single run. Afterward, multiple base classifiers built on the obtained feature subsets are combined into a stronger classifier which is applied to classify the omics data. Experimental results on real-world omics datasets demonstrate that the proposed feature selection method can stably select compact feature subsets and obtain promising classification performance.