Improving multi-label contrastive learning by leveraging label distribution

Multilabel learning (MLL), as a hot topic in the field of machine learning, has attracted wide attention from many scholars due to its ability to express output space polysemy. In recent years, a large number of achievements about MLL have emerged. Among these achievements, there are several typical issues worthy of attention. Firstly, the correlation among labels plays a key role in improving MLL model training process. Many MLL algorithms try to fully and effectively use the correlation among labels to improve the performance. Secondly, existing MLL evaluation metrics, which is different from those in binary classification, often reflects the generalization performance of MLL classifiers in some aspects. How to choose metrics in algorithms to improve their generalization performance and fairness is another issue that should be concerned. Thirdly, in many practical MLL applications, there are many unlabeled instances due to their labeling cost in training datasets. How to use the wealth information contained in the correlation among unlabeled instances may contribute to reducing of the labeling cost in MLL and improving performance. Fourthly, labels assigned to instances may not be equally descriptive in many applications. How to describe the importance of each label in output space to an instance has become one of research points that many scholars have paid attention to in recent years. This paper reviews the MLL-related research results of correlation among labels, evaluation metric, multilabel semisupervised learning, and label distribution learning (LDL) from a theoretical and algorithmic perspective. Finally, the related research work on MLL is summarized and discussed.

As we all know, the multi-label learning is faced with the "dimension disaster", and the label distribution learning is actually faced with the same problem. Therefore, it is necessary to perform some appropriate pre-processing on the data before the label distribution learning, such as feature selection. Nowadays, researchers have proposed many feature selection algorithms for multi-label learning, while feature selection algorithms for label distribution learning are few. The difference between label distribution learning and multi-label learning is that in the existing multi-label classification methods, the importance of each label is considered to be the same, but in actual cases, different labels has different importance when they are describing the same instance, that is, there is an imbalance between the labels. Therefore, this paper proposed a feature selection algorithm based on information measures in fuzzy rough set theory for label distribution learning. The mutual information in fuzzy rough set theory is used to calculate the correlation between features and labels, and then a feature subset is selected by correlation maximization and redundancy minimization strategy. Finally, the experimental results demonstrate that the proposed feature selection algorithm is able to select valid subsets of features.

In the task of multimodal emotion recognition with multi-label learning (MER-MULTI), leveraging the correlation between discrete and dimensional emotions is crucial for improving the model's performance. However, there may be a mismatch between the feature distributions of the training set and the testing set, which could result in the trained model's inability to adapt to the correlations between labels in the testing set. Therefore, a significant challenge in MER-MULTI is how to match the feature distributions of the training set and testing set samples. To tackle this issue, we propose a method called Label Distribution Adaptation for MER-MULTI. More specifically, by adapting the label distribution between the training set and testing set to remove training samples that do not match the features of the testing set. This can enhance the model's performance and generalization on testing data, enabling it to better capture the correlations between labels. Furthermore, to alleviate the difficulty of model training and inference, we design a novel loss function called Multi-label Emotion Joint Learning Loss (MEJL), which combines the correlations between discrete and dimensional emotions. Specifically, through contrastive learning, we transform the shared feature distribution of multiple labels into a space where discrete and dimensional emotions are consistent. This facilitates the model in learning the relationships between discrete and dimensional emotions. Finally, we have evaluated the proposed method, which has achieved second place in the MER-MULTI task of the MER 2023 Challenge.

Label distribution learning (LDL) can more accurately represent the degree of correlation between labels and samples than multi-label learning. However, LDL usually has limited available label data, which is not conducive to training deep learning models. Data augmentation refers to methods for solving limited data problems by introducing unobserved data or latent variables to increase the size and quality of the training dataset. In this paper, we augment the dataset by mapping the features and label distributions of the generated samples to the same subspace, and using the generator to learn the distribution of the original data in this space. First, we use the encoder and generator to extract effective information from sample features and label distributions, respectively. Second, we randomly fuse the existing label distribution to generate a new label distribution, then map it to the subspace and restore the corresponding features through the decoder. Finally, these generated samples are mixed with the original training set to train a model for predicting label distribution. Experimental results on nine real-world datasets show that our proposed algorithm can improve the performance of deep learning models to a certain extent.

Feature selection based on label distribution and fuzzy mutual information

Label distribution learning (LDL) to characterize the importance of different labels by label distribution has achieved good results in many application fields. LDL can learn more semantic information from the data than multi-label learning, however, most of the data in practical applications are single-label annotated or multi-label annotated, lacking the complete label distribution information suitable for label distribution learning. Thus, label enhancement (LE) is proposed to recover the label distributions from the logical labels. In this paper, we propose a new label enhancement method using inter-example correlation information that can automatically learn label correlations from data and jointly learn model and label correlations in a unified learning framework. Moreover, we also exploit the feature correlations constraining the model in the proposed method, which solves the problem that existing label enhancement algorithms cannot fully utilize the label information to improve the model performance. The experimental results on several real-world data sets validate the effectiveness of our method.KeywordsLabel enhancementLabel distribution learningLabel correlationFeature correlationLow rank

Label Distribution Learning (LDL), as a more general learning setting than generic single-label and multi-label learning, has been commonly used in computer vision and many other applications. To date, existing LDL approaches are designed and applied to data without considering the interdependence between instances. In this paper, we propose a Graph Label Distribution Learning (GLDL) framework, which explicitly models three types of relationships: instance-instance, label-label, and instance-label, to learn the label distribution for networked data. A label-label network is learned to capture label-to-label correlation, through which GLDL can accurately learn label distributions for nodes. Dual graph convolution network (GCN) Co-training with heterogeneous message passing ensures two GCNs, one focusing on instance-instance relationship and the other one targeting label-label correlation, are jointly trained such that instance-instance relationship can help induce label-label correlation and vice versa. Our theoretical study derives the error bound of GLDL. For verification, four benchmark datasets with label distributions for nodes are created using common graph benchmarks. The experiments show that considering dependency helps learn better label distributions for networked data, compared to state-of-the-art LDL baseline. In addition, GLDL not only outperforms simple GCN and graph attention networks (GAT) using distribution loss but is also superior to its variant considering label-label relationship as a static network. GLDL and its benchmarks are the first research endeavors to address LDL for graphs. Code and benchmark data are released for public access.

Existing feature selection methods mainly target single-label learning and multi-label learning, and only a few algorithms are optimized for label distribution learning. In label distribution learning, the associated labels of each sample have different levels of importance. Therefore, multi-label feature selection algorithms cannot be directly applied to label distribution learning. Discretizing label distribution data into multi-label data will cause part of the supervision information to be lost. In most practical applications of label distribution learning, the feature space is undefined, and the features are in the form of flow features. To solve this problem, this paper applies fuzzy rough set theory and applies the flow feature framework to propose a dynamic label distribution feature selection algorithm that handles flow features. Experimental results show that the proposed method is more effective than six state-of-the-art feature selection algorithms on 12 datasets with respect to six representative evaluation metrics.

In multi-label learning, label-specific feature learning can effectively avoid some ineffectual features that interfere with the classification performance of the model. However, most of the existing label-specific feature learning algorithms improve the performance of the model for classification by constraining the solution space through label correlation. The non-equilibrium of the label distribution not only leads to some spurious correlations mixed in with the calculated label correlations but also diminishes the performance of the classification model. Causal learning can improve the classification performance and robustness of the model by capturing real causal relationships from limited data. Based on this, this paper proposes a causality-driven intra-class non-equilibrium label-specific features learning, named CNSF. Firstly, the causal relationship between the labels is learned by the Peter-Clark algorithm. Secondly, the label density of all instances is calculated by the intra-class non-equilibrium method, which is used to relieve the non-equilibrium distribution of original labels. Then, the correlation of the density matrix is calculated using cosine similarity and combined with causality to construct the causal density correlation matrix, to solve the problem of spurious correlation mixed in the label correlation obtained by traditional methods. Finally, the causal density correlation matrix is used to induce label-specific feature learning. Compared with eight state-of-the-art multi-label algorithms on thirteen datasets, the experimental results prove the reasonability and effectiveness of the algorithms in this paper.

In multi-label image classification, each image is always associated with multiple labels and labels are usually correlated with each other. The intrinsic relation among labels can definitely contribute to classifier training. However, most previous studies on active learning for multi-label image classification purely mine label correlation based on observed label distribution. They ignore the mapping relation between examples and their labels. This mapping relation also implicates label relationship. Ignoring the mapping relation leads to an uncomprehensive label correlation estimation and results in a bad performance for classification. In this paper, we propose a novel multi-label active learning with low-rank mapping for image classification, called LMMAL, to solve this issue. More precisely, we train a low-rank mapping matrix to signify the mapping relation between the feature space and the label space of a certain multi-label dataset. Using this low-rank mapping relation, we exploit a full label correlation. Subsequently, an effective sampling strategy is designed by integrating this potential information with uncertainty to select the most informative example-label pairs. In addition, we extend LMMAL with automatic labeling (denoted as AL-LMMAL) to further reduce the annotation workload of active learning. Empirical results demonstrate the effectiveness of our approaches.

Multi-label active learning for image classification has attracted great attention over recent years and a lot of relevant works are published continuously. However, there still remain some problems that need to be solved, such as existing multi-label active learning algorithms do not reflect on the cleanness of sample data and their ways on label correlation mining are defective. For one thing, sample data is usually contaminated in reality, which disturbs the estimation of data distribution and further hinders the model training. For another, previous approaches for label relationship exploration are purely based on the observed label distribution of an incomplete training set, which cannot provide sufficiently efficient information. To address these issues, we propose a novel adaptive low-rank multi-label active learning algorithm, called LRMAL. Specifically, we first use low-rank matrix recovery to learn an effective low-rank feature representation from the noisy data. In a subsequent sampling phase, we make use of its superiorities to evaluate the general informativeness of each unlabeled example-label pair. Based on an intrinsic mapping relation between the example space and the label space of a certain multi-label dataset, we recover the incomplete labels of a training set for a more comprehensive label correlation mining. Furthermore, to reduce the redundancy among the selected example-label pairs, we use a diversity measurement to diversify the sampled data. Finally, an effective sampling strategy is developed by integrating these two aspects of potential information with uncertainty based on an adaptive integration scheme. Experimental results demonstrate the effectiveness of our approach.

Although multi-label learning can handle problems involving label ambiguity, it is not suitable for applications where the overall distribution of the importance of the labels matters. Imbalanced data also poses challenges to classification applications. In this paper, we propose a novel learning paradigm named Label Distribution Learning on Imbalanced Data (LDLID) for such applications, which is based on the improved Kullback-Leibler Divergence and uses the adaptive step size to update the gradient. The experimental results show that LDLID achieves better performance in comparison with four widely-used algorithms on four imbalanced datasets.

Exploiting asymmetric influence between instances for label enhancement

Attribute and label distribution driven multi-label active learning

Label distribution learning (LDL) is a novel multi-label learning paradigm proposed in recent years for solving label ambiguity. Existing approaches typically exploit label correlations globally to improve the effectiveness of label distribution learning, by assuming that the label correlations are shared by all instances. However, different instances may share different label correlations, and few correlations are globally applicable in real-world applications. In this paper, we propose a new label distribution learning algorithm by exploiting sample correlations locally (LDL-SCL). To encode the influence of local samples, we design a local correlation vector for each instance based on the clustered local samples. Then we predict the label distribution for an unseen instance based on the original features and the local correlation vector simultaneously. Experimental results demonstrate that LDL-SCL can effectively deal with the label distribution problems and perform remarkably better than the state-of-the-art LDL methods.