Feature selection for multi-label text data: An ensemble approach using geometric mean aggregation

Due to the existence of text data, multi-label (ML) text classification is an essential task in machine learning. Feature selection is an essential and effective preprocess to enhance the learning process. Choosing a Multi-Label Feature Selection (MLFS) algorithm is the most basic, critical, and sensitive choice in ML classification operations. If this choice is based on a criterion, it cannot be attributed to always being sound. Choosing the best algorithm must be evaluated using several different criteria to be examined from different aspects. In this article, we turn the issue into an election and use the Weighted Borda Count method for voting. We do the voting in three stages continuously so that a subset of different features does the voting. In the second stage, voting of different methods is done with six criteria, and each criterion selects the methods in order of priority from the beginning to the end. Voting steps 1 and 2 are performed on eighteen text datasets used. Finally, in the final voting stage, the methods are evaluated and voted on by different text datasets. The final result of the voting in the third stage shows the desired MLFS methods based on their performance from beginning to end. According to the experiments performed and the results obtained, it can be seen that the selection of the algorithm based on several different criteria and considering the overall performance of the algorithm will be better than the selection based on one criterion.

Insurance purchasing has been receiving increasing attention nowadays. Facing various insurance products, customers prefer the convincing recommendation by machine learning algorithms. The amount of personal information explosively grows due to the development of network and storage. Redundant features within the personal information may lead to reduced prediction accuracy and low efficiency. As a result, it is important to select effective features for multi-product prediction. Many current multi-label feature selection methods ignore the correlation between either features or labels. Thus, the selected features cannot comprehensively represent the leading factors of each label. In this paper, we propose a Multi-label Entropy-based Feature selection (MEF) method, aiming to provide a clear and reasonable explanation for insurance purchase prediction. Our method considers the relationship between features and removes redundant features. We compare our MEF model with 3 previously proposed multi-label feature selection methods on the Insurance Company Case datasets from CoIL Challenge 2000. The experiment results show that the MEF algorithm outperforms other methods with fewer features by 6 evaluation indexes and can effectively improve the classification performance of the multi-label classifier.

Recently, multilabel classification has received significant attention during the past years. A multilabel classification approach called coupled k-nearest neighbors algorithm for multilabel classification (called here as CK-STC) reported in the literature exploits coupled label similarities between the labels and provides improved performance [Liu and Cao in A Coupled k-Nearest Neighbor Algorithm for Multi-label Classification, pp. 176–187, 2015]. A multilabel feature selection is presented in Li et al. [Multi-label Feature Selection via Information Gain, pp. 346–355, 2014] and called as FSVIG here. FSVIG uses information gain that shows better performance when used with ML-NB, ML-kNN, and RandSvm when compared with existing multilabel feature selection algorithms.This paper investigates the performance of FSVIG when used with CK-STC and compares its performance with other multilabel feature selection algorithms available in MULAN using standard multilabel datasets. Experimental results show that FSVIG when used with CK-STC provides better performance in terms of average precision and one-error.

Robust feature selection using label enhancement and β-precision fuzzy rough sets for multilabel fuzzy decision system

Feature selection has been an important issue in machine learning and data mining, and is unavoidable when confronting with high‐dimensional data. With the advent of multilabel (ML) datasets and their vast applications, feature selection methods have been developed for dimensionality reduction and improvement of the classification performance. In this work, we provide a comprehensive review of the existing multilabel feature selection (ML‐FS) methods, and categorize these methods based on different perspectives. As feature selection and data classification are closely related to each other, we provide a review on ML learning algorithms as well. Also, to facilitate research in this field, a section is provided for setup and benchmarking that presents evaluation measures, standard datasets, and existing software for ML data. At the end of this survey, we discuss some challenges and open problems in this field that can be pursued by researchers in future. WIREs Data Mining Knowl Discov 2018, 8:e1240. doi: 10.1002/widm.1240This article is categorized under: Technologies > Data Preprocessing

A label-specific multi-label feature selection algorithm based on the Pareto dominance concept

Feature selection based on label distribution and fuzzy mutual information

Enhanced multi-label feature selection considering label-specific relevant information

Ant-TD: Ant colony optimization plus temporal difference reinforcement learning for multi-label feature selection

Multi-label feature selection attracts considerable attention from multi-label learning. Information theory-based multi-label feature selection methods intend to select the most informative features and reduce the uncertain amount of information of labels. Previous methods regard the uncertain amount of information of labels as constant. In fact, as the classification information of the label set is captured by features, the remaining uncertainty of each label is changing dynamically. In this paper, we categorize labels into two groups: One contains the labels with few remaining uncertainty, which means that most of classification information with respect to the labels has been obtained by the already-selected features; another group contains the labels with extensive remaining uncertainty, which means that the classification information of these labels is neglected by already-selected features. Feature selection aims to select the new features that are highly relevant to the labels in the second group. Existing methods do not distinguish the difference between two label groups and ignore the dynamic change amount of information of labels. To this end, a Relevancy Ratio is designed to clarify the dynamic change amount of information of each label under the condition of the already-selected features. Afterward, a Weighted Feature Relevancy is defined to evaluate the candidate features. Finally, a new multi-label feature selection method based on Weighted Feature Relevancy (WFRFS) is proposed. The experiments obtain encouraging results of WFRFS in comparison with six multi-label feature selection methods on thirteen real-world data sets.

Multi-label feature selection, which is an efficient and effective pre-processing step in machine learning and data mining, can select a feature subset that contains more contributions for multi-label classification while improving the performance of the classifiers. In real-world applications, an instance may be associated with multiple related labels with different relative importances, and the process of obtaining different features usually requires different costs, containing money, and time, etc. However, most existing works with regard to multi-label feature selection do not take into consideration the above two critical issues simultaneously. Therefore, in this paper, we exploit the idea of neighborhood granularity to enhance the traditional logical labels into label distribution forms to excavate the deeper supervised information hidden in multi-label data, and further consider the effect of the test cost under three different distributions, simultaneously. Motivated by these issues, a novel test cost multi-label feature selection algorithm with label enhancement and neighborhood granularity is designed. Moreover, the proposed algorithm is tested upon ten publicly available benchmark multi-label datasets with six widely-used metrics from two different aspects. Finally, two groups of experimental results demonstrate that the proposed algorithm achieves the satisfactory and superior performance over other four state-of-the-art comparing algorithms, and it is effective for improving the learning performance and decreasing the total test costs of the selected feature subset.

In classification, feature selection (FS) removes irrelevant and redundant features to improve the classification performance. Most of the existing FS approaches are designed specifically for single-label classification, wherein each instance is associated to a single class label. However, many real-world problems have more than one class label for each instance, also known as multi-label classification (MLC). Label-to-label interactions in MLC information presents a new challenge for FS, whereas single-label FS methods typically only consider feature-to-feature (redundancy) and feature-to-label (relevancy) information. Sparsity-based FS is an efficient and effective method for MLC which has demonstrated its ability in considering label-to-label interactions. However, existing sparsity-based approaches can be trapped at local optima due to their gradient-based search. In addition, they cannot consider different trade-off between the two objectives of FS: to minimise the number of features and to maximise the classification performance. Evolutionary multi-objective optimisation (EMO) can address the above issues, but EMO has not been applied to achieve sparsity-based FS for MLC. This paper proposes a novel sparsity-based representation, objective function, and local search method to achieve sparsity-based EMO FS for MLC. The experimental results indicate improved efficiency and effectiveness in the MLC performance in comparison to existing state-of-the-art multi-label feature selection methods.

Multi-label feature selection based on the division of label topics

Recently, multilabel classification algorithms play an increasingly significant role in data mining and machine learning. However, some existing mutual information-based algorithms ignore the influence of the proportions of labels on the correlation degree between features and label sets. Besides, the correlation degree of label sets cannot be accurately measured in most traditional ReliefF algorithms, and the repeated calculation arises from the division of heterogeneous neighbors. To overcome these shortcomings, this paper proposes a multilabel feature selection method using mutual information and improved multilabel ReliefF (ML-ReliefF). First, the proportion of each label is calculated in label space and combined with the mutual information of features and labels to construct a novel correlation degree between features and label sets to preprocess multilabel datasets, which is used to reduce runtime of ML-ReliefF. Second, the mutual information of label sets is introduced into improving accuracy of the correlation degree among label sets. Furthermore, two types of correlation degree for label sets based on ML-ReliefF are developed to divide similar and heterogeneous samples more clearly. Third, a divided method of heterogeneous neighbors is presented to effectively avoid the repeated calculation in ML-ReliefF, and a novel method of feature weighting based on ML-ReliefF is constructed to evaluate the importance of features. Finally, a multilabel feature selection algorithm based on mutual information and ML-ReliefF for multilabel classification is designed to improve the performance of multilabel classification. Experiments under fourteen multilabel datasets show the effectiveness of our algorithm and improve the classification performance for multilabel datasets.

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures