An Effective Multilabel Classification Using Feature Selection

Text datasets have many terms, which decrease the classification accuracy. According to the high-dimensional text data, there are more challenges for these methods. Each classification method has strengths and weaknesses in its feature selection function. Therefore, ensembling should be used for better classifications and exploitation of strengths. In this paper, for the first time, we have presented an ensemble multi-label (ML) feature selection method for the text datasets using the Geometric-Mean (GM) aggregation approach. For this purpose, we have used four multi-label feature selection (MLFS) algorithms with different structures to achieve a good result. Then, the performance and results obtained by the GM method are compared with the four algorithms and based on the six classification criteria on three ML datasets with text domains. According to the obtained outputs, it is possible to realize the ability of the GMA (proposed method) method in using strengths and ignoring weaknesses in the path of feature selection, and be more accurate. Finally, according to experiments and obtained results, superiority of the GMA over other methods can be well seen.

Due to the existence of text data, multi-label (ML) text classification is an essential task in machine learning. Feature selection is an essential and effective preprocess to enhance the learning process. Choosing a Multi-Label Feature Selection (MLFS) algorithm is the most basic, critical, and sensitive choice in ML classification operations. If this choice is based on a criterion, it cannot be attributed to always being sound. Choosing the best algorithm must be evaluated using several different criteria to be examined from different aspects. In this article, we turn the issue into an election and use the Weighted Borda Count method for voting. We do the voting in three stages continuously so that a subset of different features does the voting. In the second stage, voting of different methods is done with six criteria, and each criterion selects the methods in order of priority from the beginning to the end. Voting steps 1 and 2 are performed on eighteen text datasets used. Finally, in the final voting stage, the methods are evaluated and voted on by different text datasets. The final result of the voting in the third stage shows the desired MLFS methods based on their performance from beginning to end. According to the experiments performed and the results obtained, it can be seen that the selection of the algorithm based on several different criteria and considering the overall performance of the algorithm will be better than the selection based on one criterion.

Recently, multilabel classification algorithms play an increasingly significant role in data mining and machine learning. However, some existing mutual information-based algorithms ignore the influence of the proportions of labels on the correlation degree between features and label sets. Besides, the correlation degree of label sets cannot be accurately measured in most traditional ReliefF algorithms, and the repeated calculation arises from the division of heterogeneous neighbors. To overcome these shortcomings, this paper proposes a multilabel feature selection method using mutual information and improved multilabel ReliefF (ML-ReliefF). First, the proportion of each label is calculated in label space and combined with the mutual information of features and labels to construct a novel correlation degree between features and label sets to preprocess multilabel datasets, which is used to reduce runtime of ML-ReliefF. Second, the mutual information of label sets is introduced into improving accuracy of the correlation degree among label sets. Furthermore, two types of correlation degree for label sets based on ML-ReliefF are developed to divide similar and heterogeneous samples more clearly. Third, a divided method of heterogeneous neighbors is presented to effectively avoid the repeated calculation in ML-ReliefF, and a novel method of feature weighting based on ML-ReliefF is constructed to evaluate the importance of features. Finally, a multilabel feature selection algorithm based on mutual information and ML-ReliefF for multilabel classification is designed to improve the performance of multilabel classification. Experiments under fourteen multilabel datasets show the effectiveness of our algorithm and improve the classification performance for multilabel datasets.

Feature selection based on label distribution and fuzzy mutual information

Feature selection has been an important issue in machine learning and data mining, and is unavoidable when confronting with high‐dimensional data. With the advent of multilabel (ML) datasets and their vast applications, feature selection methods have been developed for dimensionality reduction and improvement of the classification performance. In this work, we provide a comprehensive review of the existing multilabel feature selection (ML‐FS) methods, and categorize these methods based on different perspectives. As feature selection and data classification are closely related to each other, we provide a review on ML learning algorithms as well. Also, to facilitate research in this field, a section is provided for setup and benchmarking that presents evaluation measures, standard datasets, and existing software for ML data. At the end of this survey, we discuss some challenges and open problems in this field that can be pursued by researchers in future. WIREs Data Mining Knowl Discov 2018, 8:e1240. doi: 10.1002/widm.1240This article is categorized under: Technologies > Data Preprocessing

A label-specific multi-label feature selection algorithm based on the Pareto dominance concept

Multi-label classification aims to associate multiple labels to a given data/object instance to better describe them. Multi-label data sets are common in a lot of emerging application areas like: Text/Multimedia classification, Bio-Informatics, Medical image annotations and Computer Vision to name a few. There is a growing interest in efficient and accurate multi-label classification. There are two major approaches to perform multi-label classification (i) problem transformation methods and (ii) algorithm adaptation methods. In algorithm adaptation, the traditional classification algorithms are modified to handle multi-label data sets. One classification algorithm which is often modified to do multi-label classification is k- nearest neighbor (kNN). k-nearest neighbor is popular due to its simplicity, easy to implement and seamlessly adaptability. Despite its merits it has several drawbacks like: sensitivity of noisy data, missing values and outliers; feature scaling and often becoming inaccurate for large overlapping solution space. In this paper, a modification to kNN method is suggested for multi-label classification with three improvement strategies (i) selection of local example w.r.t. unknown example – the motivation for this comes from the fact that local and relevant space is vital for the improvement in multi-label classification; (ii) Splitting the input space into multiple sub-spaces for optimal label estimation – the motivation is to estimate label accurately in the presence of noisy labels; And (iii) selection of labels using Mean Average Precision (MAP) estimates – here our motivation is to utilize the training data effectively to maximize the hidden distribution and optimal parameters for the method. The proposed method is implemented and compared with state-of-the-art approaches based on kNN or similar approaches that effectively select and optimize relevant spaces for multi-label classification. Evaluation based on multiple metrics like Hamming loss, Precision/Recall and F-measure are used for evaluation. The suggested approach performed much better than the state-of-the-art on the datasets with strong label cardinalities.

Multi-label feature selection, which is an efficient and effective pre-processing step in machine learning and data mining, can select a feature subset that contains more contributions for multi-label classification while improving the performance of the classifiers. In real-world applications, an instance may be associated with multiple related labels with different relative importances, and the process of obtaining different features usually requires different costs, containing money, and time, etc. However, most existing works with regard to multi-label feature selection do not take into consideration the above two critical issues simultaneously. Therefore, in this paper, we exploit the idea of neighborhood granularity to enhance the traditional logical labels into label distribution forms to excavate the deeper supervised information hidden in multi-label data, and further consider the effect of the test cost under three different distributions, simultaneously. Motivated by these issues, a novel test cost multi-label feature selection algorithm with label enhancement and neighborhood granularity is designed. Moreover, the proposed algorithm is tested upon ten publicly available benchmark multi-label datasets with six widely-used metrics from two different aspects. Finally, two groups of experimental results demonstrate that the proposed algorithm achieves the satisfactory and superior performance over other four state-of-the-art comparing algorithms, and it is effective for improving the learning performance and decreasing the total test costs of the selected feature subset.

Robust feature selection using label enhancement and β-precision fuzzy rough sets for multilabel fuzzy decision system

A novel random fast multi-label deep forest classification algorithm

In the last few years, several new tools addressing maturity level management have been proposed, e.g. diagnostic assessment questionnaires (DAQ). In practice, the usage of questionnaires presents some drawbacks related to subjectivity, time cost, and applicant bias. Moreover, the questionnaires may present a large number of questions, as well as part of them redundant. Another important fact of real-life application of DAQs concerns the usage of multiple questionnaires, increasing the shortcoming impacts. To pave the way to a more convenient tool to support and facilitate the achievement of organizational strategies and objectives, we proposed an intelligent reduction of DAQs by the use of single-label and multilabel feature selection. In this paper, we reduced four DAQs (Risk Management, Infrastructure, Governance and Service Catalogs) with our proposal in comparison to different feature selection algorithms (χ2, Information Gain, Random Forest Importance and ReliefF). The reduction was driven by a machine learning prediction model towards ensuring the new subset of question grounded in the same obtained score result. Results showed that removing irrelevant and/or redundant question it was possible to increase the model fitting even reducing about one-third of the questions with the same predictive capacity.

Insurance purchasing has been receiving increasing attention nowadays. Facing various insurance products, customers prefer the convincing recommendation by machine learning algorithms. The amount of personal information explosively grows due to the development of network and storage. Redundant features within the personal information may lead to reduced prediction accuracy and low efficiency. As a result, it is important to select effective features for multi-product prediction. Many current multi-label feature selection methods ignore the correlation between either features or labels. Thus, the selected features cannot comprehensively represent the leading factors of each label. In this paper, we propose a Multi-label Entropy-based Feature selection (MEF) method, aiming to provide a clear and reasonable explanation for insurance purchase prediction. Our method considers the relationship between features and removes redundant features. We compare our MEF model with 3 previously proposed multi-label feature selection methods on the Insurance Company Case datasets from CoIL Challenge 2000. The experiment results show that the MEF algorithm outperforms other methods with fewer features by 6 evaluation indexes and can effectively improve the classification performance of the multi-label classifier.

Multi-label feature selection based on the division of label topics

Multi-label feature selection via similarity constraints with non-negative matrix factorization

In multi-label classification a datapoint can be labelled with more than one class at the same time. A common but trivial approach to multi-label classification is to train individual binary classifiers per label, but the performance can be improved by considering associations between the labels, and algorithms like classifier chains and RAKEL do this effectively. Like most machine learning algorithms, however, these approaches require accurate hyperparameter tuning, a computationally expensive optimisation problem. Tuning is important to train a good multi-label classifier model. There is a scarcity in the literature of effective multi-label classification approaches that do not require extensive hyperparameter tuning. This paper addresses this scarcity by proposing CascadeML, a multi-label classification approach based on cascade neural network that takes label associations into account and requires minimal hyperparameter tuning. The performance of the CasecadeML approach is evaluated using 10 multi-label datasets and compared with other leading multi-label classification algorithms. Results show that CascadeML performs comparatively with the leading approaches but without a need for hyperparameter tuning.