Feature Selection for High Dimensional Data: An Evolutionary Filter Approach

In classification and clustering problems, feature selection techniques can be used to reduce the dimensionality of the data and increase the performances. However, feature selection is a challenging task, especially when hundred or thousands of features are involved. In this framework, we present a new approach for improving the performance of a filter-based genetic algorithm. The proposed approach consists of two steps: first, the available features are ranked according to a univariate evaluation function; then the search space represented by the first M features in the ranking is searched using a filter-based genetic algorithm for finding feature subsets with a high discriminative power.

Feature selection in high dimensional data: A specific preordonnances-based memetic algorithm

Classification and high dimensional data are the challenging problems in machine learning. In this paper, we carry out an empirical study of model selection for high dimensional data. We select three single classifiers, i.e., decision tree, support vector machine, and naive bayes, and two ensemble methods, i.e., bagging and random forest, to compare the performance of these five methods through real high dimensional data. Six real gene datasets with high correlation are used in this empirical study. Experimental results have illustrated that ensemble methods are better than single classifiers in terms of accuracy metric. In which, decision tree model is the weakest one, and bagging is the best model. The performance of random forest is much more efficient in terms of computing time metrics compared with bagging. From this empirical study, we can conclude that ensemble methods are an effective solution for high dimensional data.KeywordsHigh dimensional dataSupervised learningClassification

High dimension and complexity of network high-dimensional data lead to poor feature selection effect network high-dimensional data. To effectively solve this problem, feature selection algorithms for high-dimensional network data based on supervised discriminant projection (SDP) have been designed. The sparse representation problem of high-dimensional network data is transformed into an Lp norm optimization problem, and the sparse subspace clustering method is used to cluster high-dimensional network data. Dimensionless processing is carried out for the clustering processing results. Based on the linear projection matrix and the best transformation matrix, the dimensionless processing results are reduced by combining the SDP. The sparse constraint method is used to achieve feature selection of high-dimensional data in the network, and the relevant feature selection results are obtained. The experimental findings demonstrate that the suggested algorithm can effectively cluster seven different types of data and converges when the number of iterations approaches 24. The F1 value, recall, and precision are all kept at high levels. High-dimensional network data feature selection accuracy on average is 96.9%, and feature selection time on average is 65.1 milliseconds. The selection effect for network high-dimensional data features is good.

Improved Salp Swarm Algorithm based on opposition based learning and novel local search algorithm for feature selection

Boosting classification accuracy using an efficient stochastic optimization technique for feature selection in high-dimensional data

Based on benchmarking various methods and analyzing multiple real scRNA-seq datasets, a computational platform/workflow and a set of tips for best practices are developed for analyzing causal interactions or relationships in single cells.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

This study is aimed at assessing the usefulness of different feature selection and classification methodologies in the context of sleep apnea hypopnea syndrome (SAHS) detection. Feature extraction, selection and classification stages were applied to analyze blood oxygen saturation (SaO2) recordings in order to simplify polysomnography (PSG), the gold standard diagnostic methodology for SAHS. Statistical, spectral and nonlinear measures were computed to compose the initial feature set. Principal component analysis (PCA), forward stepwise feature selection (FSFS) and genetic algorithms (GAs) were applied to select feature subsets. Fisher's linear discriminant (FLD), logistic regression (LR) and support vector machines (SVMs) were applied in the classification stage. Optimum classification algorithms from each combination of these feature selection and classification approaches were prospectively validated on datasets from two independent sleep units. FSFS + LR achieved the highest diagnostic performance using a small feature subset (4 features), reaching 83.2% accuracy in the validation set and 88.7% accuracy in the test set. Similarly, GAs + SVM also achieved high generalization capability using a small number of input features (7 features), with 84.2% accuracy on the validation set and 84.5% accuracy in the test set. Our results suggest that reduced subsets of complementary features (25% to 50% of total features) and classifiers with high generalization ability could provide high-performance screening tools in the context of SAHS.

The classification of microarray data has positive significance for the judgment of cancer and the determination of clinical programs. However, the high dimensionality and small sample characteristics of the microarray data has brought classification a difficult problem. Aiming at the feature selection problem in microarray data classification, a feature selection algorithm based on artificial bee colony algorithm and genetic algorithm is proposed to solve the dimensionality disaster problem in microarray data classification. Finally, the feature subsets obtained by the algorithm are combined with the SVM(Support Vector Machine)classifier to apply to the six published microarray datasets. The experimental results show that the feature subsets obtained based on the proposed scheme can significantly improve the classification performance.

Because of the poor performance of the Random Forest algorithm in processing the classification of high-dimensional unbalanced data, a Hybrid Samping&Feature Selection Random Forest optimization strategy (Hybrid Samping&Feature Selection Random Forest (HF_RF) is proposed in this paper. First, from the data level, the high-dimensional unbalanced data set is preprocessed by SMOTE algorithm combined with random undersampling to achieve balanced unbalanced data. At the same time, the clustering algorithm is combined with SMOTE algorithm to improve the processing ability of the algorithm for negative samples; On the algorithm level, through the Relief F algorithm, different weight values are given to the preprocessed high-dimensional data, irrelevant and redundant features are eliminated, and high-dimensional data is reduced for dimensionality; Finally, the weighted voting principle is used to further elevate the predictive performance of HF_RF. The experimental results show that compared with the traditional algorithm, the proposed algorithm has higher indicators when dealing with high-dimensional unbalanced data, which proves that the HF_RF proposed in this paper is The correctness of the algorithm and its effectiveness in improving the classification performance of high-dimensional unbalanced data.

In this paper, we demonstrate the need for dimensionality reduction to mitigate model overfitting on the nontrivial problem of gender classification from digital images. In this study we explore four feature selection schemes using Genetic Algorithm, Memetic Algorithms, and Random Forest, which are fed to a nonlinear support vector machine (SVM) for final classification. The performance of the model (feature) selection approaches are evaluated against two distinct datasets of facial images: FG-NET which contains toddlers to seniors and the UIUC-PAL which contains faces of adults up to seniors. This work demonstrates that feature selection can, and does, improve performance of an SVM based gender classification system significantly.

Multidate imagery and satellite image derivatives such as vegetation and texture indices have been reported to improve image classification. However, the increase in additional predictor variables has also resulted in high data dimensionality and redundancy. Feature selection and extraction can be used to reduce high data dimensionality and minimize redundancy. The purpose of this chapter is to test whether feature selection can improve image classification. In this chapter, image classification will be performed using two different approaches. First, image classification is performed using the random forests (RF) classifier and multiple data sets (that consist of multidate Landsat 5 TM imagery, and vegetation and texture indices). Second, image classification is performed using the RF classifier with feature selection and multiple data sets. While the tutorial exercises indicate that feature selection did not improve image classification accuracy, it reduced the number of predictor variables.

Random Forest is a supervised classification method based on bagging (Bootstrap aggregating) Breiman and random selection of features. The choice of features randomly assigned to the Random Forest makes it possible that the selected feature is not necessarily informative. So it is necessary to select features in the Random Forest. The purpose of choosing this feature is to select an optimal subset of features that contain valuable information in the hope of accelerating the performance of the Random Forest method. Mainly for the execution of high-dimensional datasets such as the Parkinson, CNAE-9, and Urban Land Cover dataset. The feature selection is done using the Correlation-Based Feature Selection method, using the BestFirst method. Tests were carried out 30 times using the K-Cross Fold Validation value of 10 and dividing the dataset into 70% training and 30% testing. The experiments using the Parkinson dataset obtained a time difference of 0.27 and 0.28 seconds faster than using the Random Forest method without feature selection. Likewise, the trials in the Urban Land Cover dataset had 0.04 and 0.03 seconds, while for the CNAE-9 dataset, the difference time was 2.23 and 2.81 faster than using the Random Forest method without feature selection. These experiments showed that the Random Forest processes are faster when using the first feature selection. Likewise, the accuracy value increased in the two previous experiments, while only the CNAE-9 dataset experiment gets a lower accuracy. This research’s benefits is by first performing feature selection steps using the Correlation-Base Feature Selection method can increase the speed of performance and accuracy of the Random Forest method on high-dimensional data.

The high dimensionality of electronic nose data increases the difficulty of their use in classification models. Reducing this high dimensionality helps reduce variable numbers, improve classification accuracy, and reduce computation time and sensor cost. In this research, the Cyranose 320 electronic nose, which was used for apple defect detection, was optimized by selecting only the most relevant of its internal 32 sensors using different selection methods. Two robust heuristic optimization algorithms, genetic algorithm (GA) and covariance matrix adaptation evolutionary strategy (CMAES), were applied and compared. Although both algorithms searched the optimal sensors resulting in a best classification error rate of 4.4%, the average classification error rate of CMA over 30 random seed runs was 5.0% (s.d.=0.006) which was better than 5.2% (s.d.=0.004) from the GA. The final optimal solution sets obtained by integer GA showed that including more sensors did not guarantee better classification performance. The best reduction in classification error rate was 10% while the number of sensors was reduced 78%. This study provided a robust and efficient optimization approach to reduce high data dimensionality of the electronic nose data, which substantially improved electronic nose performance in apple defect detection while potentially reducing the overall electronic nose cost for future specific applications.