A robust graph-based semi-supervised sparse feature selection method

Orthogonally constrained matrix factorization for robust unsupervised feature selection with local preserving

In many real applications, the collected data is usually high dimensionality of feature space and no labels are given, which will result in the “curse of dimensionality” problem and bring the challenge for the feature selection. Although tremendous efforts have been made in unsupervised feature selection, the existing methods neglect the robustness of jointly feature selection and clustering. In this paper, we present a robust simultaneous feature selection and clustering method. By introducing a robust loss function, the unsupervised linear discriminant analysis methods with two different strategies (the ratio trace and the trace ratio) are incorporated into this loss function, which can guarantee the selected features with robustness. Then two novel simultaneous unsupervised feature selection and clustering objective functions are presented by the proposed loss function equipped with $\ell_{2,1}$-norm. Two alternative iterative optimization algorithms are developed to solve these two objective functions. The comparative experimental results conducted on eight UCI datasets show that the proposed methods are better than other competing methods.

To recognize expressions accurately, facial expression systems require robust feature extraction and feature selection methods. In this paper, a normalized mutual information based feature selection technique is proposed for FER systems. The technique is derived from an existing method, that is, the max-relevance and min-redundancy (mRMR) method. We, however, propose to normalize the mutual information used in this method so that the domination of the relevance or of the redundancy can be eliminated. For feature extraction, curvelet transform is used. After the feature extraction and selection the feature space is reduced by employing linear discriminant analysis (LDA). Finally, hidden Markov model (HMM) is used to recognize the expressions. The proposed FER system (CNF-FER) is validated using four publicly available standard datasets. For each dataset, 10-fold cross validation scheme is utilized. CNF-FER outperformed the existing well-known statistical and state-of-the-art methods by achieving a weighted average recognition rate of 99 % across all the datasets.

Feature selection is an important method of data dimensionality reduction widely used in machine learning. In this framework, the sparse representation based feature selection methods are very attractive. This is because of the nature of these methods which try to represent a data with as less as possible non-zero coefficients. In deep neural networks, a very high dimensional feature space is usually existed. In such a situation, one can take advantages of the feature selection approaches into account. In this paper, first, three sparse feature selection methods are compared. The Sparse Group Lasso (SGL) algorithm is one of the adopted approaches. This method is theoretically very well-organized and leads to good results for man-made features. The most important property of this method is that it highly induces the sparsity to the data. A main step of the SGL method is the features grouping step. In this paper, a k-means clustering based method is applied for grouping of the features. Our experimental results show that this sparse representation based method leads to very successful results in deep neural networks.

In recent years, feature selection (FS) methods based on sparsity have been extensively investigated due to their high performance. These methods solve the FS problem mainly by introducing some kinds of sparsity regularization terms. However, recent existing feature selection algorithms combine sparsity regularization with simple linear loss function, which may lead to deficient in performance. To this end, we propose a fresh and robust feature selection method that combines the structured sparsity regularization, i.e., \(\ell _{2,0}\)-norm regularization, with the Softmax model to find a stable row-sparse solution, where we can select the features in group according to the solution of the projected matrix, and the classification performance can be improved by Softmax. Extensive experiments on six different datasets indicate that our method can obtain better or comparable classification performance by using fewer features compared with other advanced sparsity-based FS methods.Keywords\(\L _{2 , 0}\)-norm regularizationFeature selectionStructured sparsitySoftmax

When the number of labeled examples is limited, traditional supervised feature selection techniques often fail due to sample selection bias or unrepresentative sample problem. To solve this, semi-supervised feature selection techniques exploit the statistical information of both labeled and unlabeled examples in the same time. However, the results of semi-supervised feature selection can be at times unsatisfactory, and the culprit is on how to effectively use the unlabeled data. Quite different from both supervised and semi-supervised feature selection, we propose a ldquohybridrdquoframework based on graph models. We first apply supervised methods to select a small set of most critical features from the labeled data. Importantly, these initial features might otherwise be missed when selection is performed on the labeled and unlabeled examples simultaneously. Next,this initial feature set is expanded and corrected with the use of unlabeled data. We formally analyze why the expected performance of the hybrid framework is better than both supervised and semi-supervised feature selection. Experimental results demonstrate that the proposed method outperforms both traditional supervised and state-of-the-art semi-supervised feature selection algorithms by at least 10% inaccuracy on a number of text and biomedical problems with thousands of features to choose from. Software and dataset is available from the authors.

Neurodynamics-driven holistic approaches to semi-supervised feature selection

Exploiting unlabeled data for battery state-of-health estimation using transformer-LSTM neural network with semi-supervised learning

Local sparse discriminative feature selection

The explosive growth of big data poses a processing challenge for predictive systems in terms of both data size and its dimensionality. Generating features from text often leads to many thousands of sparse features rarely taking non-zero values. In this work we propose a very fast and robust feature selection method that is optimised with the Naive Bayes classifier. The method takes advantage of the sparse feature representation and uses diversified backward-forward greedy search to arrive with the highly competitive solution at the minimum processing time. It promotes the paradigm of shifting the complexity of predictive systems away from the model algorithm, but towards careful data preprocessing and filtering that allows to accomplish predictive big data tasks on a single processor despite billions of data examples nominally exposed for processing. This method was applied to the AAIA Data Mining Competition 2014 concerned with predicting human injuries as a result of fire incidents based on nearly 12000 risk factors extracted from thousands of fire incident reports and scored the second place with the predictive accuracy of 96%.

The most common deadly neoplasm of the female reproductive organ is exhibited by ovarian cancer. The microarray data offers valuable diagnostic and prognostic data for ovarian cancer studies. DNA microarray technology contains high dimensions with small sample size. In high- dimensional microarray data, all genes are not related to hereditary disease. However, in diagnosing disease, these uninformative genes might lead to the misleading results. Identifying a small set of features is very essential as, an efficient and robust feature selection method can achieve significant classification accuracy. This paper proposes an Integrated Feature selection method utilizing Evolutionary Non- Dominated Radial Slots Based Algorithm (ENORA) and Consistency subset evaluation (CSE) stochastic optimization algorithms that can select the highly significant features. This proposed study used ENORA to decrease redundant features and CSE was used to discover the best smallest significant feature subset from the redundant features. The Artificial Neural Network classification model was utilized to evaluate Ovarian Cancer benchmark dataset. The outcomes of the experiment were contrasted with the conventional feature selection model of other literature. Particularly, this proposed approach attained 99.20% higher classification accuracy rate than previous study for ovarian cancer datasets with a minimal sized predictive feature subset.

Software defect prediction models are used for predicting high risk software components. Feature selection has significant impact on the prediction performance of the software defect prediction models since redundant and unimportant features make the prediction model more difficult to learn. Ensemble feature selection has recently emerged as a new methodology for enhancing feature selection performance. This paper proposes a new multi-criteria-decision-making (MCDM) based ensemble feature selection (EFS) method. This new method is termed as MCDM-EFS. The proposed method, MCDM-EFS, first generates the decision matrix signifying the feature’s importance score with respect to various existing feature selection methods. Next, the decision matrix is used as the input to well-known MCDM method TOPSIS for assigning a final rank to each feature. The proposed approach is validated by an experimental study for predicting software defects using two classifiers K-nearest neighbor (KNN) and naïve bayes (NB) over five open-source datasets. The predictive performance of the proposed approach is compared with existing feature selection algorithms. Two evaluation metrics – nMCC and G-measure are used to compare predictive performance. The experimental results show that the MCDM-EFS significantly improves the predictive performance of software defect prediction models against other feature selection methods in terms of nMCC as well as G-measure.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

Robust feature selection using label enhancement and β-precision fuzzy rough sets for multilabel fuzzy decision system

The paper proposes a hybrid approach to reduce dimension in text classification problems, to overcome the issue of Curse of Dimensionality. This hybrid approach is a combination of Feature Selection (FS) and Feature Extraction (FE) methods, considering different aspects of feature relevance, to effectively reduce the dimension in large text datasets. It prevents feature selection biased in favor of a particular FS method. Many FS methods like Term Variance, Document Frequency, Information Gain, Shannons Entropy measure, Mean-Median and Mean Absolute Difference, were implemented and a comparative study was made on their performance when implemented in a hybrid system. The features selected by the individual FS methods are merged using three approaches, namely, Union, Intersection and Modified Union. The sub lists of features further undergo Feature Extraction by PCA, and the reduced feature sub list is clustered with k-means. Finally, the sentiment-score of the individual clusters are calculated using SentiWordNet database which gives the polarity of the data. The experiments were conducted on the benchmark datasets namely Reuters-21,578 and Classic4. The performance evaluation of the system made using the measures like precision, recall, f-score and accuracy shows that the proposed method has improved performance compared to its competitive methods.