Robust Unsupervised Feature Selection on Networked Data

Orthogonally constrained matrix factorization for robust unsupervised feature selection with local preserving

Robust feature selection using label enhancement and β-precision fuzzy rough sets for multilabel fuzzy decision system

High-dimensional data expands the spatial dimension, leading to increased computational complexity and reduced generalization performance. Microarray data classification, such as diagnosing diseases like cancer, involves complex dimensions due to their genetic and biological information. To address this issue, dimension reduction is essential for these data sets. The main goal of this chapter is to provide a method for dimension reduction and classification of genetic data sets. The proposed approach comprises multiple stages. Initially, various feature ranking methods are combined to improve the robustness and stability of the feature selection process. A hybrid ranking method, which incorporates gene interactions, is integrated with a wrapper method. Subsequently, a support vector machine (SVM) is employed for classification. To address class imbalance in the training data, a solution is implemented before feeding the data into the SVM classifier. The experimental outcomes of the proposed approach, tested on five microarray databases, indicate robust feature selection with a metric ranging from 0.70 to 0.88. Additionally, the classification accuracy falls within the range of 91-96%.

With the continuous development of Internet technology, data gradually present a complicated and high-dimensional trend. These high-dimensional data have a large number of redundant features and irrelevant features, which bring great challenges to the existing machine learning algorithms. Feature selection is one of the important research topics in the fields of machine learning, pattern recognition and data mining, and it is also an important means in the data preprocessing stage. Feature selection is to look for the optimal feature subset from the original feature set, which would improve the classification accuracy and reduce the machine learning time. The traditional feature selection algorithm tends to ignore the kind of feature which has a weak distinguishing capacity as a monomer, whereas the feature group’s distinguishing capacity is strong. Therefore, a new dynamic interaction feature selection (DIFS) algorithm is proposed in this paper. Initially, under the theoretical framework of interactive information, it redefines the relevance, irrelevance and redundancy of the features. Secondly, it offers the computational formulas for calculating interactive information. Finally, under the eleven data sets of UCI and three different classifiers, namely, KNN, SVM and C4.5, the DIFS algorithm increases the classification accuracy of the FullSet by 3.2848% and averagely decreases the number of features selected by 15.137. Hence, the DIFS algorithm can not only identify the relevance feature effectively, but also identify the irrelevant and redundant features. Moreover, it can effectively improve the classification accuracy of the data sets and reduce the feature dimensions of the data sets.

Feature selection techniques are enormously applied in a variety of data analysis tasks in order to reduce the dimensionality. According to the type of learning, feature selection algorithms are categorized to: supervised or unsupervised. In unsupervised learning scenarios, selecting features is a much harder problem, due to the lack of class labels that would facilitate the search for relevant features. The selecting feature difficulty is amplified when the data is corrupted by different noises. Almost all traditional unsupervised feature selection methods are not robust against the noise in samples. These approaches do not have any explicit mechanism for detaching and isolating the noise thus they can not produce an optimal feature subset. In this article, we propose an unsupervised approach for feature selection on noisy data, called Robust Independent Feature Selection (RIFS). Specifically, we choose feature subset that contains most of the underlying information, using the same criteria as the Independent component analysis (ICA). Simultaneously, the noise is separated as an independent component. The isolation of representative noise samples is achieved using factor oblique rotation whereas noise identification is performed using factor pattern loadings. Extensive experimental results over divers real-life data sets have showed the efficiency and advantage of the proposed algorithm.

A data-driven method approach for prediction of coal seam gas content combining feature selection and machine learning

In a conventional supervised learning paradigm, each data instance is associated with one single class label. Multilabel learning differs in the way that data instances may belong to multiple concepts simultaneously, which naturally appear in a variety of high impact domains, ranging from bioinformatics and information retrieval to multimedia analysis. It targets leveraging the multiple label information of data instances to build a predictive learning model that can classify unlabeled instances into one or multiple predefined target classes. In multilabel learning, even though each instance is associated with a rich set of class labels, the label information could be noisy and incomplete as the labeling process is both time consuming and labor expensive, leading to potential missing annotations or even erroneous annotations. The existence of noisy and missing labels could negatively affect the performance of underlying learning algorithms. More often than not, multilabeled data often has noisy, irrelevant, and redundant features of high dimensionality. The existence of these uninformative features may also deteriorate the predictive power of the learning model due to the curse of dimensionality. Feature selection, as an effective dimensionality reduction technique, has shown to be powerful in preparing high-dimensional data for numerous data mining and machine-learning tasks. However, a vast majority of existing multilabel feature selection algorithms either boil down to solving multiple single-labeled feature selection problems or directly make use of the imperfect labels to guide the selection of representative features. As a result, they may not be able to obtain discriminative features shared across multiple labels. In this article, to bridge the gap between a rich source of multilabel information and its blemish in practical usage, we propose a novel noise-resilient multilabel informed feature selection framework (MIFS) by exploiting the correlations among different labels. In particular, to reduce the negative effects of imperfect label information in obtaining label correlations, we decompose the multilabel information of data instances into a low-dimensional space and then employ the reduced label representation to guide the feature selection phase via a joint sparse regression framework. Empirical studies on both synthetic and real-world datasets demonstrate the effectiveness and efficiency of the proposed MIFS framework.

In the present state of affairs, the motive behind every educational organization is to uplift the academic achievement of students. Educational data mining (EDM) is an upward field of research, and it is very helpful for academic institutions to predict the academic performance of the students. Educational datasets are the basis of various predictive models. Quality of these models can be improved by using feature selection (FS). To get the required benefits from the available data, there must be some tools for analysis and prediction. In lieu of the above, machine learning/data mining are most suitable. In educational data mining, for better accuracy of prediction models’ and quality of various educational datasets, feature selection (FS) plays a vital role. Feature selection (FS) algorithms abolish inappropriate information from the repositories of educational background so that performance of classifier in terms of accuracy could be increased and the same could be used for better decision. In lieu of the above, a best feature selection algorithm must be selected. In this paper, two filter selection approaches namely correlation feature selection (CFS) and wrapper-based feature selection have been used to demonstrate the importance of selection of a feature subset for a classification problem. The present paper aims to find the detailed investigation of filter feature selection algorithms along with the classification algorithms on a given dataset. We found result with numerous numbers of features from various Feature selection algorithms and classifiers which will help the researcher to discover the most excellent mixture of filter feature selection algorithms and its associated classifiers. The result indicates that SMO and J48 have the highest accuracy measures with the correlation feature selection algorithms, while Naïve Bayes has the highest accuracy measures with the wrapper subset feature selection algorithms for predicting high, medium and low grade for the students.

Robust microarray data feature selection using a correntropy based distance metric learning approach

Feature selection has attracted considerable attention due to the wide application of multi-label learning. However, previous methods do not fully consider the relationship between feature sets and label sets but devote attention to either of them. Furthermore, conventional multi-label learning utilizes logical labels to estimate relevance between feature sets and label sets so that the importance of corresponding labels cannot be well reflected. Additionally, numerous irrelevant and redundant labels degrade the classification performance of models. To this end, we propose a multi-label feature selection method named Robust multi-label Feature Selection with shared Label Enhanced (RLEFS). First, we obtain a robust label enhancement term by reconstructing labels from logical labels to numerical labels and imposing \(l_{2,1}\)-norm onto the label enhancement term. Second, RLEFS utilizes the robust label enhancement term to share the similar latent semantic structure between feature matrix and label matrix. Third, local structure is considered to ensure the consistency of label information during the feature selection process. Finally, we integrate the above terms into one joint learning framework, and then, a simple but effective optimization method with provable convergence is proposed to solve RLEFS. Experimental results demonstrate the classification superiority of RLEFS in comparison with seven state-of-the-art methods.

PGC-PTSD

Feature (or variable) selection is the process of identifying the minimal set of features with the highest predictive performance on the target variable of interest. Numerous feature selection algorithms have been developed over the years, but only few have been implemented in R and made publicly available R as packages while offering few options. The R package MXM offers a variety of feature selection algorithms, and has unique features that make it advantageous over its competitors: a) it contains feature selection algorithms that can treat numerous types of target variables, including continuous, percentages, time to event (survival), binary, nominal, ordinal, clustered, counts, left censored, etc; b) it contains a variety of regression models that can be plugged into the feature selection algorithms (for example with time to event data the user can choose among Cox, Weibull, log logistic or exponential regression); c) it includes an algorithm for detecting multiple solutions (many sets of statistically equivalent features, plain speaking, two features can carry statistically equivalent information when substituting one with the other does not effect the inference or the conclusions); and d) it includes memory efficient algorithms for high volume data, data that cannot be loaded into R (In a 16GB RAM terminal for example, R cannot directly load data of 16GB size. By utilizing the proper package, we load the data and then perform feature selection.). In this paper, we qualitatively compare MXM with other relevant feature selection packages and discuss its advantages and disadvantages. Further, we provide a demonstration of MXM’s algorithms using real high-dimensional data from various applications.

Exploring features associated with the clinical outcome of interest is a rapidly advancing area of research. However, with contemporary sequencing technologies capable of identifying over thousands of genes per sample, there is a challenge in constructing efficient prediction models that balance accuracy and resource utilization. To address this challenge, researchers have developed feature selection methods to enhance performance, reduce overfitting, and ensure resource efficiency. However, applying feature selection models to survival analysis, particularly in clinical datasets characterized by substantial censoring and limited sample sizes, introduces unique challenges. We propose a robust ensemble feature selection approach integrated with group Lasso to identify compelling features and evaluate its performance in predicting survival outcomes. Our approach consistently outperforms established models across various criteria through extensive simulations, demonstrating low false discovery rates, high sensitivity, and high stability. Furthermore, we applied the approach to a colorectal cancer dataset from The Cancer Genome Atlas, showcasing its effectiveness by generating a composite score based on the selected genes to correctly distinguish different subtypes of the patients. In summary, our proposed approach excels in selecting impactful features from high-dimensional data, yielding better outcomes compared to contemporary state-of-the-art models.

A robust graph-based semi-supervised sparse feature selection method

PurposeThis study aimed to select robust features against lung motion in a phantom study and use them as input to feature selection algorithms and machine learning classifiers in a clinical study to predict the lymphovascular invasion (LVI) of non-small cell lung cancer (NSCLC). The results of robust features were also compared with conventional techniques without considering the robustness of radiomic features.MethodsAn in-house developed lung phantom was developed with two 22mm lesion sizes based on a clinical study. A specific motor was built to simulate motion in two orthogonal directions. Lesions of both clinical and phantom studies were segmented using a Fuzzy C-means-based segmentation algorithm. After inducing motion and extracting 105 radiomic features in 4 feature sets, including shape, first-, second-, and higher-order statistics features from each region of interest (ROI) of the phantom image, statistical analyses were performed to select robust features against motion. Subsequently, these robust features and a total of 105 radiomic features were extracted from 126 clinical data. Various feature selection (FS) and multiple machine learning (ML) classifiers were implemented to predict the LVI of NSCLC, followed by comparing the results of predicting LVI using robust features with common conventional techniques not considering the robustness of radiomic features.ResultsOur results demonstrated that selecting robust features as input to FS algorithms and ML classifiers surges the sensitivity, which has a gentle negative effect on the accuracy and the area under the curve (AUC) of predictions compared with commonly used methods in 12 of 15 outcomes. The top performance of the LVI prediction was achieved by the NB classifier and RFE FS without considering the robustness of radiomic features with 95% area under the curve of AUC, 67% accuracy, and 100% sensitivity. Moreover, the top performance of the LVI prediction using robust features belonged to the NB classifier and Boruta feature selection with 92% AUC, 86% accuracy, and 100% sensitivity.ConclusionRobustness over various influential factors is critical and should be considered in a radiomic study. Selecting robust features is a solution to overcome the low reproducibility of radiomic features. Although setting robust features against motion in a phantom study has a minor negative impact on the accuracy and AUC of LVI prediction, it boosts the sensitivity of prediction to a large extent.