Extending version-space theory to multi-label active learning with imbalanced data

Multi-label learning draws great interests in many real world applications. It is a highly costly task to assign many labels by the oracle for one instance. Meanwhile, it is also hard to build a good model without diagnosing discriminative labels. Can we reduce the label costs and improve the ability to train a good model for multi-label learning simultaneously? Active learning addresses the less training samples problem by querying the most valuable samples to achieve a better performance with little costs. In multi-label active learning, some researches have been done for querying the relevant labels with less training samples or querying all labels without diagnosing the discriminative information. They all cannot effectively handle the outlier labels for the measurement of uncertainty. Since maximum correntropy criterion (MCC) provides a robust analysis for outliers in many machine learning and data mining algorithms, in this paper, we derive a robust multi-label active learning algorithm based on an MCC by merging uncertainty and representativeness, and propose an efficient alternating optimization method to solve it. With MCC, our method can eliminate the influence of outlier labels that are not discriminative to measure the uncertainty. To make further improvement on the ability of information measurement, we merge uncertainty and representativeness with the prediction labels of unknown data. It cannot only enhance the uncertainty but also improve the similarity measurement of multi-label data with labels information. Experiments on benchmark multi-label data sets have shown a superior performance than the state-of-the-art methods.

Multi-label learning is an effective framework for learning with objects that have multiple semantic labels, and has been successfully applied into many real-world tasks. In contrast with traditional single-label learning, the cost of labeling a multi-label example is rather high, thus it becomes an important task to train an effectivemulti-label learning model with as few labeled examples as possible. Active learning, which actively selects the most valuable data to query their labels, is the most important approach to reduce labeling cost. In this paper, we propose a novel approach MADM for batch mode multi-label active learning. On one hand, MADM exploits representativeness and diversity in both the feature and label space by matching the distribution between labeled and unlabeled data. On the other hand, it tends to query predicted positive instances, which are expected to be more informative than negative ones. Experiments on benchmark datasets demonstrate that the proposed approach can reduce the labeling cost significantly.

Multi-label active learning for image classification has attracted great attention over recent years and a lot of relevant works are published continuously. However, there still remain some problems that need to be solved, such as existing multi-label active learning algorithms do not reflect on the cleanness of sample data and their ways on label correlation mining are defective. For one thing, sample data is usually contaminated in reality, which disturbs the estimation of data distribution and further hinders the model training. For another, previous approaches for label relationship exploration are purely based on the observed label distribution of an incomplete training set, which cannot provide sufficiently efficient information. To address these issues, we propose a novel adaptive low-rank multi-label active learning algorithm, called LRMAL. Specifically, we first use low-rank matrix recovery to learn an effective low-rank feature representation from the noisy data. In a subsequent sampling phase, we make use of its superiorities to evaluate the general informativeness of each unlabeled example-label pair. Based on an intrinsic mapping relation between the example space and the label space of a certain multi-label dataset, we recover the incomplete labels of a training set for a more comprehensive label correlation mining. Furthermore, to reduce the redundancy among the selected example-label pairs, we use a diversity measurement to diversify the sampled data. Finally, an effective sampling strategy is developed by integrating these two aspects of potential information with uncertainty based on an adaptive integration scheme. Experimental results demonstrate the effectiveness of our approach.

In multi-label image classification, each image is always associated with multiple labels and labels are usually correlated with each other. The intrinsic relation among labels can definitely contribute to classifier training. However, most previous studies on active learning for multi-label image classification purely mine label correlation based on observed label distribution. They ignore the mapping relation between examples and their labels. This mapping relation also implicates label relationship. Ignoring the mapping relation leads to an uncomprehensive label correlation estimation and results in a bad performance for classification. In this paper, we propose a novel multi-label active learning with low-rank mapping for image classification, called LMMAL, to solve this issue. More precisely, we train a low-rank mapping matrix to signify the mapping relation between the feature space and the label space of a certain multi-label dataset. Using this low-rank mapping relation, we exploit a full label correlation. Subsequently, an effective sampling strategy is designed by integrating this potential information with uncertainty to select the most informative example-label pairs. In addition, we extend LMMAL with automatic labeling (denoted as AL-LMMAL) to further reduce the annotation workload of active learning. Empirical results demonstrate the effectiveness of our approaches.

International audience

Multilabel learning (MLL), as a hot topic in the field of machine learning, has attracted wide attention from many scholars due to its ability to express output space polysemy. In recent years, a large number of achievements about MLL have emerged. Among these achievements, there are several typical issues worthy of attention. Firstly, the correlation among labels plays a key role in improving MLL model training process. Many MLL algorithms try to fully and effectively use the correlation among labels to improve the performance. Secondly, existing MLL evaluation metrics, which is different from those in binary classification, often reflects the generalization performance of MLL classifiers in some aspects. How to choose metrics in algorithms to improve their generalization performance and fairness is another issue that should be concerned. Thirdly, in many practical MLL applications, there are many unlabeled instances due to their labeling cost in training datasets. How to use the wealth information contained in the correlation among unlabeled instances may contribute to reducing of the labeling cost in MLL and improving performance. Fourthly, labels assigned to instances may not be equally descriptive in many applications. How to describe the importance of each label in output space to an instance has become one of research points that many scholars have paid attention to in recent years. This paper reviews the MLL-related research results of correlation among labels, evaluation metric, multilabel semisupervised learning, and label distribution learning (LDL) from a theoretical and algorithmic perspective. Finally, the related research work on MLL is summarized and discussed.

Multilabel image classification has been a hot topic in the field of computer vision and image understanding in recent years. To achieve better classification performance with fewer labeled images, multilabel active learning is used for this scenario. Several active learning methods have been proposed for multilabel image classification. However, all of them assume that either all training images have complete labels or label correlations are given at the beginning. These two assumptions are unrealistic. In fact, it is very difficult to obtain complete labels for each example, in particular when the size of labels in a multilabel dataset is very large. Typically, only partial labels are available. This is one type of “weak label” problem. To solve this weak label problem inside multilabel active learning, this paper proposes a novel solution called AE-WLMAL. AE-WLMAL explores conditional label correlations on the weak label problem with the help of input features and then utilizes label correlations to construct a unified sampling strategy and evaluate the informativeness of each example-label pair in a multilabel dataset for active sampling. In addition, a pruning strategy is adopted to further improve its computation efficiency. Moreover, AE-WLAML exploits label correlations to infer labels for unlabeled images, which further reduces human labeling cost. Our experimental results on seven real-world datasets show that AE-WLMAL consistently outperforms existing approaches.

Tongue diagnosis characterization is a key research issue in the development of Traditional Chinese Medicine (TCM). Many kinds of information, such as tongue body color, coat color and coat thickness, can be reflected from a tongue image. That is, tongue images are multi-label data. However, traditional supervised learning is used to model single-label data. In this paper, multi-label learning is applied to the tongue image classification. Color features and texture features are extracted after separation of tongue coat and body, and multi-label learning algorithms are used for classification. Results showed LEAD (Multi-Label Learning by Exploiting Label Dependency), a multi-label learning algorithm demonstrating to exploit correlations among labels, is superior to the other multi-label algorithms. At last, the iteration algorithm is used to set an optimal threshold for each label to improve the results of LEAD. In this paper, we have provided an effective way for computer aided TCM diagnosis.

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures

A structured perspective of volumes on active learning

Automated annotation of protein function is challenging. As the number of sequenced genomes rapidly grows, the vast majority of proteins can only be annotated computationally. Nature often brings several domains together to form multi-domain and multi-functional proteins with a vast number of possibilities, and each domain may fulfill its own function independently or in a concerted manner with its neighbors. Thus, it is evident that the protein function prediction problem is naturally and inherently Multi-Instance Multi-Label (MIML) learning tasks. Based on the state-of-the-art MIML algorithm MIMLNN, we propose a novel ensemble MIML learning framework EnMIMLNN and design three algorithms for this task by combining the advantage of three kinds of Hausdorff distance metrics. Experiments on seven real-world organisms covering the biological three-domain system, i.e., archaea, bacteria, and eukaryote, show that the EnMIMLNN algorithms are superior to most state-of-the-art MIML and Multi-Label learning algorithms.

In recent years, machine learning has been vastly used in many scientific experiments to analyze and predict the data [1]. Since there are many types of machine learning and each of them has additional further branches, it's necessary to understand which one can predict or summarize the data with more accuracy. This paper mainly focuses on traditional machine learning and active learning. To compare the effects of these two types of machine learning, the paper uses a dataset about using metal oxide semiconductor (MOX) to measure the concentration of carbon monoxide (CO). In the first part, the paper tests the accuracy of predicting CO concentration with 3 types of traditional machine learning: Decision Tree, Random Forest and K-nearest neighbors. In the second part, the paper chooses the traditional machine learning which has the best performance in the first part and compares the accuracy of it with the active learning on predicting CO concentration. The result is that the classification accuracy of Random Forest is 0.625, which is the highest among them. And active learning is generally better than traditional machine learning when training samples are small, and they have a similar accuracy when the training sample is enough.

For automatic object detection tasks, large amounts of training images are usually labeled to achieve more reliable training of the object classifiers; this is cost-expensive since it requires hiring professionals to label large-scale training images. When a large number of object classes come into view, the issue of obtaining a large enough amount of the labeled training images becomes more critical. There are three potential solutions to reduce the burden for image labeling: (1) allowing people to provide the object labels loosely at the image level rather than at the object level (e.g., loosely-tagged images without identifying the exact object locations in the images); (2) harnessing large-scale collaboratively-tagged images that are available on the Internet; and, (3) developing new machine learning algorithms that can directly leverage large-scale collaboratively- or loosely-tagged images for achieving more effective training of a large number of object classifiers. Based on these observations, a multi-task multi-label multiple instance learning (MTML-MIL) algorithm is developed in this paper by leveraging both interobject correlations and large-scale loosely-labeled images for object classifier training. By seamlessly integrating multi-task learning, multi-label learning, and multiple instance learning, our MTML-MIL algorithm can achieve more accurate training of a large number of inter-related object classifiers (where an object network is constructed for determining the inter-related learning tasks directly in the feature space rather than in the label space). Our experimental results have shown that our MTML-MIL algorithm can achieve higher detection accuracy rates for automatic object detection.

The prognostic and health management (PHM) of rolling bearings has been a popular research area. Since bearing fault is inevitable during degradation, how to improve the PHM performance based on both degradation states and fault types is still an open problem. In this study, two multilabel learning algorithms are proposed for PHM of rolling bearings, named personalized binary relevance (PBR) and hierarchical multilabel K-nearest neighbor (HML-KNN), respectively. Degradation states and fault types are used as the labels of the bearing data so that each sample has a corresponding label sequence, that is to say, the PHM problem is converted to a multilabel learning problem. Both algorithms have a personalized search process, which can not only help samples build a personalized model to improve classification accuracy but also solve the problem of data imbalance between labels. At the same time, the two algorithms also have their own characteristics and focus on different application situations. The PBR algorithm has faster modeling speed, more flexible use, and replaceable subclassifiers. HML-KNN is a high-order algorithm with global information analysis capabilities through the hierarchical processing of data and the conversion of label information. Both methods have achieved good enough results in the XJTU-SY bearing dataset. In order to illustrate the practicality of the algorithm, the experimental part further increases the difficulty. Using only a single faulty sample as the training set to determine the type of compound fault, the algorithms also show high performance. In the era of industrial big data, the depth of data mining and the design of algorithm models will help us better manage equipment health.

Existing feature selection methods mainly target single-label learning and multi-label learning, and only a few algorithms are optimized for label distribution learning. In label distribution learning, the associated labels of each sample have different levels of importance. Therefore, multi-label feature selection algorithms cannot be directly applied to label distribution learning. Discretizing label distribution data into multi-label data will cause part of the supervision information to be lost. In most practical applications of label distribution learning, the feature space is undefined, and the features are in the form of flow features. To solve this problem, this paper applies fuzzy rough set theory and applies the flow feature framework to propose a dynamic label distribution feature selection algorithm that handles flow features. Experimental results show that the proposed method is more effective than six state-of-the-art feature selection algorithms on 12 datasets with respect to six representative evaluation metrics.