Multi-label Learning with Highly Incomplete Data via Collaborative Embedding

Multi-label learning draws great interests in many real world applications. It is a highly costly task to assign many labels by the oracle for one instance. Meanwhile, it is also hard to build a good model without diagnosing discriminative labels. Can we reduce the label costs and improve the ability to train a good model for multi-label learning simultaneously? Active learning addresses the less training samples problem by querying the most valuable samples to achieve a better performance with little costs. In multi-label active learning, some researches have been done for querying the relevant labels with less training samples or querying all labels without diagnosing the discriminative information. They all cannot effectively handle the outlier labels for the measurement of uncertainty. Since maximum correntropy criterion (MCC) provides a robust analysis for outliers in many machine learning and data mining algorithms, in this paper, we derive a robust multi-label active learning algorithm based on an MCC by merging uncertainty and representativeness, and propose an efficient alternating optimization method to solve it. With MCC, our method can eliminate the influence of outlier labels that are not discriminative to measure the uncertainty. To make further improvement on the ability of information measurement, we merge uncertainty and representativeness with the prediction labels of unknown data. It cannot only enhance the uncertainty but also improve the similarity measurement of multi-label data with labels information. Experiments on benchmark multi-label data sets have shown a superior performance than the state-of-the-art methods.

In a conventional supervised learning paradigm, each data instance is associated with one single class label. Multilabel learning differs in the way that data instances may belong to multiple concepts simultaneously, which naturally appear in a variety of high impact domains, ranging from bioinformatics and information retrieval to multimedia analysis. It targets leveraging the multiple label information of data instances to build a predictive learning model that can classify unlabeled instances into one or multiple predefined target classes. In multilabel learning, even though each instance is associated with a rich set of class labels, the label information could be noisy and incomplete as the labeling process is both time consuming and labor expensive, leading to potential missing annotations or even erroneous annotations. The existence of noisy and missing labels could negatively affect the performance of underlying learning algorithms. More often than not, multilabeled data often has noisy, irrelevant, and redundant features of high dimensionality. The existence of these uninformative features may also deteriorate the predictive power of the learning model due to the curse of dimensionality. Feature selection, as an effective dimensionality reduction technique, has shown to be powerful in preparing high-dimensional data for numerous data mining and machine-learning tasks. However, a vast majority of existing multilabel feature selection algorithms either boil down to solving multiple single-labeled feature selection problems or directly make use of the imperfect labels to guide the selection of representative features. As a result, they may not be able to obtain discriminative features shared across multiple labels. In this article, to bridge the gap between a rich source of multilabel information and its blemish in practical usage, we propose a novel noise-resilient multilabel informed feature selection framework (MIFS) by exploiting the correlations among different labels. In particular, to reduce the negative effects of imperfect label information in obtaining label correlations, we decompose the multilabel information of data instances into a low-dimensional space and then employ the reduced label representation to guide the feature selection phase via a joint sparse regression framework. Empirical studies on both synthetic and real-world datasets demonstrate the effectiveness and efficiency of the proposed MIFS framework.

Ensemble of kernel extreme learning machine based elimination optimization for multi-label classification

Multi-label feature selection attracts considerable attention from multi-label learning. Information theory-based multi-label feature selection methods intend to select the most informative features and reduce the uncertain amount of information of labels. Previous methods regard the uncertain amount of information of labels as constant. In fact, as the classification information of the label set is captured by features, the remaining uncertainty of each label is changing dynamically. In this paper, we categorize labels into two groups: One contains the labels with few remaining uncertainty, which means that most of classification information with respect to the labels has been obtained by the already-selected features; another group contains the labels with extensive remaining uncertainty, which means that the classification information of these labels is neglected by already-selected features. Feature selection aims to select the new features that are highly relevant to the labels in the second group. Existing methods do not distinguish the difference between two label groups and ignore the dynamic change amount of information of labels. To this end, a Relevancy Ratio is designed to clarify the dynamic change amount of information of each label under the condition of the already-selected features. Afterward, a Weighted Feature Relevancy is defined to evaluate the candidate features. Finally, a new multi-label feature selection method based on Weighted Feature Relevancy (WFRFS) is proposed. The experiments obtain encouraging results of WFRFS in comparison with six multi-label feature selection methods on thirteen real-world data sets.

Link prediction and multi-label learning on graphs are two important but challenging machine learning problems that have broad applications in diverse fields. Not only are the two problems inherently correlated and often appear concurrently, they are also exacerbated by incomplete data. We develop a novel algorithm to solve these two problems jointly under a unified framework, which helps reduce the impact of graph noise and benefits both tasks individually. We reduce multi-label learning problem into an additional link prediction task and solve both problems with marginalized denoising, which we co-regularize with Laplacian smoothing. This approach combines both learning tasks into a single convex objective function, which we optimize efficiently with iterative closed-form updates. The resulting approach performs significantly better than prior work on several important real-world applications with great consistency.

Prediction of protein subcellular localization is an important but challenging problem, particularly when proteins may simultaneously exist at, or move between, two or more different subcellular location sites. Most of the existing protein subcellular localization methods are only used to deal with the single-location proteins. In the past few years, only a few methods have been proposed to tackle proteins with multiple locations. However, they only adopt a simple strategy, that is, transforming the multilocation proteins to multiple proteins with single location, which does not take correlations among different subcellular locations into account. In this paper, a novel method named random label selection (RALS) (multilabel learning via RALS), which extends the simple binary relevance (BR) method, is proposed to learn from multilocation proteins in an effective and efficient way. RALS does not explicitly find the correlations among labels, but rather implicitly attempts to learn the label correlations from data by augmenting original feature space with randomly selected labels as its additional input features. Through the fivefold cross-validation test on a benchmark data set, we demonstrate our proposed method with consideration of label correlations obviously outperforms the baseline BR method without consideration of label correlations, indicating correlations among different subcellular locations really exist and contribute to improvement of prediction performance. Experimental results on two benchmark data sets also show that our proposed methods achieve significantly higher performance than some other state-of-the-art methods in predicting subcellular multilocations of proteins. The prediction web server is available at http://levis.tongji.edu.cn:8080/bioinfo/MLPred-Euk/ for the public usage.

Existing multi-label learning algorithms mostly assume that all class labels in an obtained data set are completely accurate. This assumption is rarely valid. As the scale of the label space increases, owing to limited domain knowledge and differences among methods of data acquisition, some examples will be unlabeled or mislabeled. The introduction of such noise results in the poor performance of trained multi-label classifiers. This paper proposes a new method for multi-label learning with label noise, called low-rank-constraint-based multi-label learning with label noise (ML-LRC), by incorporating low-rank matrix recovery and features selection into the learning process. First, we assume that two strongly correlated labels share more samples than two weakly correlated labels. Based on this assumption, the ML-LRC applies low-rank constraints to the label matrix to mine the local correlations of class labels and remove noise from the observed label matrix. Second, we believe that each class label only correlates to some features. So we control the sparsity of the coefficient matrix to filter out label-specific features, which can reduce the complexity of the model and greatly improve the performance of the classifier. The accelerated proximal gradient method and alternating direction method of multipliers are adopted to solve the problem. Experiments conducted on four benchmark multi-label datasets demonstrate the competitive performance of our proposed method compared with several state-of-the-art methods.

In multi-label learning, each object belongs to multiple class labels simultaneously. In the data explosion age, the size of data is often huge, i.e., large number of instances, features and class labels. The high dimension of both the feature and label spaces has posed great challenges to multi-label learning problems, e.g., high time and memory costs. In this paper, we propose a new framework for multi-label learning with a large number of class labels and features, i.e., M ulti- L abel L earning via F eature and L abel S pace D imension R eduction, namely MLL-FLSDR. Specifically, both the feature space and label space are reduced to low dimensional spaces respectively, in which the local structure of data points is utilized to constrain the geometrical structure on both the learned low dimensional spaces and guarantee the qualities of them. Then, an effective multi-label classifier is constructed from the low dimensional feature space to the latent label space. Last, the final prediction for new test data examples can be obtained by recovering from their prediction results in the latent label space with an encoding matrix learned in the previous stage. Extensive comparison experiments with the state-of-the-art approaches manifest the effectiveness of the proposed method MLL-FLSDR.

Multi-label double-layer learning for cross-modal retrieval

Weakly-supervised multi-label learning has attracted wide attention recently. Most existing methods deal with such problem with incomplete labels and the feature information is ideal. However, in many scenarios, the acquired features may be corrupted due to the influence of occlusion, illumination and low-resolution, and the robustness of learning methods may be reduced. To overcome the above shortcoming, we propose a novel weakly-supervised multi-label learning algorithm, where a linear self-recovery model is adopted to reconstruct observed label information. Specifically, we first decompose the acquired feature matrix into an ideal feature matrix and an outlier matrix. To adequately utilize the visual information among instances, we introduce the graph Laplacian regularization. In addition, a linear self-recovery model is adopted to reconstruct the observed label matrix. Finally, the desired model is trained on ideal feature matrix and refined label matrix. Extensive experimental results prove the robustness of the proposed framework.

The prognostic and health management (PHM) of rolling bearings has been a popular research area. Since bearing fault is inevitable during degradation, how to improve the PHM performance based on both degradation states and fault types is still an open problem. In this study, two multilabel learning algorithms are proposed for PHM of rolling bearings, named personalized binary relevance (PBR) and hierarchical multilabel K-nearest neighbor (HML-KNN), respectively. Degradation states and fault types are used as the labels of the bearing data so that each sample has a corresponding label sequence, that is to say, the PHM problem is converted to a multilabel learning problem. Both algorithms have a personalized search process, which can not only help samples build a personalized model to improve classification accuracy but also solve the problem of data imbalance between labels. At the same time, the two algorithms also have their own characteristics and focus on different application situations. The PBR algorithm has faster modeling speed, more flexible use, and replaceable subclassifiers. HML-KNN is a high-order algorithm with global information analysis capabilities through the hierarchical processing of data and the conversion of label information. Both methods have achieved good enough results in the XJTU-SY bearing dataset. In order to illustrate the practicality of the algorithm, the experimental part further increases the difficulty. Using only a single faulty sample as the training set to determine the type of compound fault, the algorithms also show high performance. In the era of industrial big data, the depth of data mining and the design of algorithm models will help us better manage equipment health.

Label distribution learning (LDL) to characterize the importance of different labels by label distribution has achieved good results in many application fields. LDL can learn more semantic information from the data than multi-label learning, however, most of the data in practical applications are single-label annotated or multi-label annotated, lacking the complete label distribution information suitable for label distribution learning. Thus, label enhancement (LE) is proposed to recover the label distributions from the logical labels. In this paper, we propose a new label enhancement method using inter-example correlation information that can automatically learn label correlations from data and jointly learn model and label correlations in a unified learning framework. Moreover, we also exploit the feature correlations constraining the model in the proposed method, which solves the problem that existing label enhancement algorithms cannot fully utilize the label information to improve the model performance. The experimental results on several real-world data sets validate the effectiveness of our method.KeywordsLabel enhancementLabel distribution learningLabel correlationFeature correlationLow rank

In recent years, the retrieval of plane geometry figures (PGFs) has attracted increasing attention in the fields of mathematics education and computer science. However, the high cost of matching complex PGF features leads to the low efficiency of most retrieval systems. This paper proposes an indirect classification method based on multi-label learning, which improves retrieval efficiency by reducing the scope of compare operation from the whole database to small candidate groups. Label correlations among PGFs are taken into account for the multi-label classification task. The primitive feature selection for multi-label learning and the feature description of visual geometric elements are conducted individually to match similar PGFs. The experiment results show the competitive performance of the proposed method compared with existing PGF retrieval methods in terms of both time consumption and retrieval quality.

Multi-label learning has been successfully applied to solve instance multi-semantics problems. Moreover, the topology information of samples is often adopted in existing works to improve the prediction performance, in which the similarity of samples is usually calculated in the entire feature space. However, in real-world applications, each label is often determined by a subset of the original features, so when we focus on different labels, the similarity of two instances may be different. In this paper, we propose a multi-label learning method by exploiting the local similarity of samples. Specifically, the smoothness assumption is applied to assume that if the feature subset is similar between samples, the corresponding label should be similar. In addition, L1 regularization is also adopted to sparse the weight coefficients when constraining the output space of the instance. The experimental results on several data sets validate the effectiveness of the proposed method.

Multi-label learning based deep transfer neural network for facial attribute classification