A Simple and Convex Formulation for Multi-label Feature Selection

Feature selection based on label distribution and fuzzy mutual information

A label-specific multi-label feature selection algorithm based on the Pareto dominance concept

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures

LSFSR: Local label correlation-based sparse multilabel feature selection with feature redundancy

Text datasets have many terms, which decrease the classification accuracy. According to the high-dimensional text data, there are more challenges for these methods. Each classification method has strengths and weaknesses in its feature selection function. Therefore, ensembling should be used for better classifications and exploitation of strengths. In this paper, for the first time, we have presented an ensemble multi-label (ML) feature selection method for the text datasets using the Geometric-Mean (GM) aggregation approach. For this purpose, we have used four multi-label feature selection (MLFS) algorithms with different structures to achieve a good result. Then, the performance and results obtained by the GM method are compared with the four algorithms and based on the six classification criteria on three ML datasets with text domains. According to the obtained outputs, it is possible to realize the ability of the GMA (proposed method) method in using strengths and ignoring weaknesses in the path of feature selection, and be more accurate. Finally, according to experiments and obtained results, superiority of the GMA over other methods can be well seen.

Insurance purchasing has been receiving increasing attention nowadays. Facing various insurance products, customers prefer the convincing recommendation by machine learning algorithms. The amount of personal information explosively grows due to the development of network and storage. Redundant features within the personal information may lead to reduced prediction accuracy and low efficiency. As a result, it is important to select effective features for multi-product prediction. Many current multi-label feature selection methods ignore the correlation between either features or labels. Thus, the selected features cannot comprehensively represent the leading factors of each label. In this paper, we propose a Multi-label Entropy-based Feature selection (MEF) method, aiming to provide a clear and reasonable explanation for insurance purchase prediction. Our method considers the relationship between features and removes redundant features. We compare our MEF model with 3 previously proposed multi-label feature selection methods on the Insurance Company Case datasets from CoIL Challenge 2000. The experiment results show that the MEF algorithm outperforms other methods with fewer features by 6 evaluation indexes and can effectively improve the classification performance of the multi-label classifier.

Multi-label learning is dedicated to learning functions so that each sample is labeled with a true label set. With the increase of data knowledge, the feature dimensionality is increasing. However, high-dimensional information may contain noisy data, making the process of multi-label learning difficult. Feature selection is a technical approach that can effectively reduce the data dimension. In the study of feature selection, the multi-objective optimization algorithm has shown an excellent global optimization performance. The Pareto relationship can handle contradictory objectives in the multi-objective problem well. Therefore, a Shapley value-fused feature selection algorithm for multi-label learning (SHAPFS-ML) is proposed. The method takes multi-label criteria as the optimization objectives and the proposed crossover and mutation operators based on Shapley value are conducive to identifying relevant, redundant and irrelevant features. The comparison of experimental results on real-world datasets reveals that SHAPFS-ML is an effective feature selection method for multi-label classification, which can reduce the classification algorithm’s computational complexity and improve the classification accuracy.

Multi-label feature selection via similarity constraints with non-negative matrix factorization

Ant-TD: Ant colony optimization plus temporal difference reinforcement learning for multi-label feature selection

Many real-world data have multiple class labels known as multi-label data, where the labels are correlated with each other, and as such, they are not independent. Since these data are usually high-dimensional, and the current multi-label feature selection methods have not been precise enough, then a new feature selection method is necessary. In this paper, for the first time, we have modeled the problem of multi-label feature selection to a bipartite graph matching process. The proposed method constructs a bipartite graph of features (as the left vertices) and labels (as the right vertices), called Feature-Label Graph (FLG), where each feature is connected to the set of labels, where the weight of the edge between each feature and label is equal to their correlation. Then, the Hungarian algorithm estimates the best matching in FLG. The selected features in each matching are sorted by weighted correlation distance and added to the ranking vector. To select the discriminative features, the proposed method considers both the redundancy of features and the relevancy of each feature to the class labels. The results indicate the superiority of the proposed method against the other methods in classification measures.

Feature selection has attracted considerable attention due to the wide application of multi-label learning. However, previous methods do not fully consider the relationship between feature sets and label sets but devote attention to either of them. Furthermore, conventional multi-label learning utilizes logical labels to estimate relevance between feature sets and label sets so that the importance of corresponding labels cannot be well reflected. Additionally, numerous irrelevant and redundant labels degrade the classification performance of models. To this end, we propose a multi-label feature selection method named Robust multi-label Feature Selection with shared Label Enhanced (RLEFS). First, we obtain a robust label enhancement term by reconstructing labels from logical labels to numerical labels and imposing \(l_{2,1}\)-norm onto the label enhancement term. Second, RLEFS utilizes the robust label enhancement term to share the similar latent semantic structure between feature matrix and label matrix. Third, local structure is considered to ensure the consistency of label information during the feature selection process. Finally, we integrate the above terms into one joint learning framework, and then, a simple but effective optimization method with provable convergence is proposed to solve RLEFS. Experimental results demonstrate the classification superiority of RLEFS in comparison with seven state-of-the-art methods.

Multi-label feature selection attracts considerable attention from multi-label learning. Information theory-based multi-label feature selection methods intend to select the most informative features and reduce the uncertain amount of information of labels. Previous methods regard the uncertain amount of information of labels as constant. In fact, as the classification information of the label set is captured by features, the remaining uncertainty of each label is changing dynamically. In this paper, we categorize labels into two groups: One contains the labels with few remaining uncertainty, which means that most of classification information with respect to the labels has been obtained by the already-selected features; another group contains the labels with extensive remaining uncertainty, which means that the classification information of these labels is neglected by already-selected features. Feature selection aims to select the new features that are highly relevant to the labels in the second group. Existing methods do not distinguish the difference between two label groups and ignore the dynamic change amount of information of labels. To this end, a Relevancy Ratio is designed to clarify the dynamic change amount of information of each label under the condition of the already-selected features. Afterward, a Weighted Feature Relevancy is defined to evaluate the candidate features. Finally, a new multi-label feature selection method based on Weighted Feature Relevancy (WFRFS) is proposed. The experiments obtain encouraging results of WFRFS in comparison with six multi-label feature selection methods on thirteen real-world data sets.

For classification of health data, we propose in this paper a fast and accurate feature selection method, FIEBIT (Feature Inclusion and Exclusion Based on Information Theory). FIEBIT selects the most relevant and non-redundant features using Conditional Mutual Information (CMU) while excluding irrelevant and redundant features according to the comparison among Individual Symmetrical Uncertainty (ISU) and Combined Symmetrical Uncertainty (CSU). Small feature subsets are selected before classification without compromising the classification accuracy. In addition, the size of the feature subset is determined automatically. Our preliminary empirical results on health data with hundreds of features suggest FIEBIT is efficient and effective in comparison with representative feature selection methods.

Robust feature selection using label enhancement and β-precision fuzzy rough sets for multilabel fuzzy decision system

Advances in data collection have resulted in an exponential growth of high-dimensional microarray datasets for binary classification in bioinformatics and medical diagnostics. These datasets generally possess many features but relatively few samples, resulting in challenges associated with the “curse of dimensionality”, such as feature redundancy and an elevated risk of overfitting. While traditional feature selection approaches, such as filter-based and wrapper-based approaches, can help to reduce dimensionality, they often struggle to capture feature interactions while adequately preserving model generalization. Therefore, this paper introduces the Adaptive Cluster-Guided Simple, Fast, and Efficient (ACG-SFE) feature selection, a hybrid approach designed to address the challenges of high-dimensional microarray data in binary classification. ACG-SFE enhances the Simple, Fast, and Efficient (SFE) evolutionary feature selection model by integrating hierarchical clustering to dynamically group correlated features based on the optimal number of clusters determined by the Silhouette index, Davies-Bouldin score, and the feature-to-observation ratio while adaptively selecting representative features within clusters using mutual information and adjusting the selection threshold through a progress factor. This hybrid filter-wrapper approach improves feature interactions, effectively minimizing redundancy and overfitting while enhancing classification performance. The proposed model is assessed against four state-of-the-art evolutionary feature selection models on 11 high-dimensional microarray datasets. Experimental results indicate that ACG-SFE effectively selects a small yet pertinent feature subset, minimizing redundancy while attaining enhanced classification accuracy and F-measure. Furthermore, its reduced RMSE between train and test accuracy substantiates its capability to reduce overfitting, outperforming comparative models. These findings establish ACG-SFE as an effective feature selection model for handling high-dimensional microarray data in binary classification, enhancing classification accuracy while selecting minimal relevant features to reduce unnecessary complexity and the risk of overfitting.