Label Distribution Learning by Exploiting Sample Correlations Locally

Nowadays, label distribution learning is among the state-of-the-art methodologies in facial age estimation. It takes the age of each facial image instance as a label distribution with a series of age labels rather than the single chronological age label that is commonly used. However, this methodology is deficient in its simple decision-making criterion: the final predicted age is only selected at the one with maximum description degree. In many cases, different age labels may have very similar description degrees. Consequently, blindly deciding the estimated age by virtue of the highest description degree would miss or neglect other valuable age labels that may contribute a lot to the final predicted age. In this paper, we propose a strategic decision-making label distribution learning algorithm (SDM-LDL) with a series of strategies specialized for different types of age label distribution. Experimental results from the most popular aging face database, FG-NET, show the superiority and validity of all the proposed strategic decision-making learning algorithms over the existing label distribution learning and other single-label learning algorithms for facial age estimation. The inner properties of SDM-LDL are further explored with more advantages.

Existing feature selection methods mainly target single-label learning and multi-label learning, and only a few algorithms are optimized for label distribution learning. In label distribution learning, the associated labels of each sample have different levels of importance. Therefore, multi-label feature selection algorithms cannot be directly applied to label distribution learning. Discretizing label distribution data into multi-label data will cause part of the supervision information to be lost. In most practical applications of label distribution learning, the feature space is undefined, and the features are in the form of flow features. To solve this problem, this paper applies fuzzy rough set theory and applies the flow feature framework to propose a dynamic label distribution feature selection algorithm that handles flow features. Experimental results show that the proposed method is more effective than six state-of-the-art feature selection algorithms on 12 datasets with respect to six representative evaluation metrics.

Label Distribution Learning (LDL), as a more general learning setting than generic single-label and multi-label learning, has been commonly used in computer vision and many other applications. To date, existing LDL approaches are designed and applied to data without considering the interdependence between instances. In this paper, we propose a Graph Label Distribution Learning (GLDL) framework, which explicitly models three types of relationships: instance-instance, label-label, and instance-label, to learn the label distribution for networked data. A label-label network is learned to capture label-to-label correlation, through which GLDL can accurately learn label distributions for nodes. Dual graph convolution network (GCN) Co-training with heterogeneous message passing ensures two GCNs, one focusing on instance-instance relationship and the other one targeting label-label correlation, are jointly trained such that instance-instance relationship can help induce label-label correlation and vice versa. Our theoretical study derives the error bound of GLDL. For verification, four benchmark datasets with label distributions for nodes are created using common graph benchmarks. The experiments show that considering dependency helps learn better label distributions for networked data, compared to state-of-the-art LDL baseline. In addition, GLDL not only outperforms simple GCN and graph attention networks (GAT) using distribution loss but is also superior to its variant considering label-label relationship as a static network. GLDL and its benchmarks are the first research endeavors to address LDL for graphs. Code and benchmark data are released for public access.

Stream label distribution learning processing via broad learning system

Label distribution learning (LDL) can more accurately represent the degree of correlation between labels and samples than multi-label learning. However, LDL usually has limited available label data, which is not conducive to training deep learning models. Data augmentation refers to methods for solving limited data problems by introducing unobserved data or latent variables to increase the size and quality of the training dataset. In this paper, we augment the dataset by mapping the features and label distributions of the generated samples to the same subspace, and using the generator to learn the distribution of the original data in this space. First, we use the encoder and generator to extract effective information from sample features and label distributions, respectively. Second, we randomly fuse the existing label distribution to generate a new label distribution, then map it to the subspace and restore the corresponding features through the decoder. Finally, these generated samples are mixed with the original training set to train a model for predicting label distribution. Experimental results on nine real-world datasets show that our proposed algorithm can improve the performance of deep learning models to a certain extent.

Label distribution learning (LDL) is a novel paradigm that describe the samples by label distribution of a sample. However, acquiring LDL dataset is costly and time-consuming, which leads to the birth of incomplete label distribution learning (IncomLDL). All the previous IncomLDL methods set the description degrees of "missing" labels in an instance to 0, but remains those of other labels unchanged. This setting is unrealistic because when certain labels are missing, the degrees of the remaining labels will increase accordingly. We fix this unrealistic setting in IncomLDL and raise a new problem: LDL with hidden labels (HidLDL), which aims to recover a complete label distribution from a real-world incomplete label distribution where certain labels in an instance are omitted during annotation. To solve this challenging problem, we discover the significance of proportional information of the observed labels and capture it by an innovative constraint to utilize it during the optimization process. We simultaneously use local feature similarity and the global low-rank structure to reveal the mysterious veil of hidden labels. Moreover, we **theoretically** give the recovery bound of our method, proving the feasibility of our method in learning from hidden labels. Extensive recovery and predictive experiments on various datasets prove the superiority of our method to state-of-the-art LDL and IncomLDL methods.

As we all know, the multi-label learning is faced with the "dimension disaster", and the label distribution learning is actually faced with the same problem. Therefore, it is necessary to perform some appropriate pre-processing on the data before the label distribution learning, such as feature selection. Nowadays, researchers have proposed many feature selection algorithms for multi-label learning, while feature selection algorithms for label distribution learning are few. The difference between label distribution learning and multi-label learning is that in the existing multi-label classification methods, the importance of each label is considered to be the same, but in actual cases, different labels has different importance when they are describing the same instance, that is, there is an imbalance between the labels. Therefore, this paper proposed a feature selection algorithm based on information measures in fuzzy rough set theory for label distribution learning. The mutual information in fuzzy rough set theory is used to calculate the correlation between features and labels, and then a feature subset is selected by correlation maximization and redundancy minimization strategy. Finally, the experimental results demonstrate that the proposed feature selection algorithm is able to select valid subsets of features.

Label distribution learning (LDL) provides a robust framework for classification tasks exhibiting intrinsic label ordinality and ambiguity, yet its application to sentiment analysis remains underexplored. This work bridges this research gap through a comprehensive investigation of LDL for sentiment classification, revealing both strengths (predictive accuracy) and limitations (calibration reliability). To address these limitations, we propose Unimodal-Weighted LDL (UW-LDL), a novel framework that: (1) enforces unimodal label distributions via ordinality-grounded regularization, and (2) implements instance-specific calibration through adaptive weighted loss functions. Extensive experiments across five diverse benchmarks—including financial-domain datasets (AFS, FP, TFNS) and other sentiment corpora (SST-5, TSE)—demonstrate that UW-LDL outperforms state-of-the-art LDL methods and calibration techniques by significant margins in both predictive accuracy and uncertainty quantification. Furthermore, UW-LDL achieves these gains with single-pass inference, offering faster computation than ensemble-based calibration methods. Visualization analyses confirm that UW-LDL produces more discriminative and linearly separable text representations than conventional LDL approaches.

Many existing studies reveal that annotation inconsistency widely exists among a variety of facial expression recognition (FER) datasets. The reason might be the subjectivity of human annotators and the ambiguous nature of the expression labels. One promising strategy tackling such a problem is a recently proposed learning paradigm called Label Distribution Learning (LDL), which allows multiple labels with different intensity to be linked to one expression. However, it is often impractical to directly apply label distribution learning because numerous existing datasets only contain one-hot labels rather than label distributions. To solve the problem, we propose a novel approach named Label Distribution Learning on Auxiliary Label Space Graphs(LDL-ALSG) that leverages the topological information of the labels from related but more distinct tasks, such as action unit recognition and facial landmark detection. The underlying assumption is that facial images should have similar expression distributions to their neighbours in the label space of action unit recognition and facial landmark detection. Our proposed method is evaluated on a variety of datasets and outperforms those state-of-the-art methods consistently with a huge margin.

Predicting Alzheimer's disease (AD) clinical scores offers a useful tool to monitor dementia progression at different time points. Previous machine learning methods usually focused on regressing the clinical scores directly but ignored the ambigu-ous information among score labels. In this study, we introduce a novel AD clinical scores prediction framework based on label distribution learning (LDL), named CSP-LDL. Notably, we first turn the clinical scores into a normal probability distribution of discrete labels to exploit uncertainty among the dementia scores. Then we learn the distribution of discrete labels by optimizing Kullback-Leibler (KL) divergence between the estimated and ground-truth distributions using a 3D CNN. Moreover, we further employ an expectation regression layer to regress clinical score value at the fine-grained level based on the predicted label distribution. Experiments on ADNI-1 and ADNI-2 datasets show that our CSP-LDL model outperforms existing state-of-the-art methods in terms of dementia regression accuracy at multiple time points using baseline structural magnetic resonance imaging (sMRI) data, demonstrating its effectiveness in early clinical diagnosis of AD.

Label distribution learning (LDL) is a new machine learning paradigm for solving label ambiguity. Since it is difficult to directly obtain label distributions, many studies are focusing on how to recover label distributions from logical labels, dubbed label enhancement (LE). Existing LE methods estimate label distributions by simply building a mapping relationship between features and label distributions under the supervision of logical labels. They typically overlook the fact that both features and logical labels are descriptions of the instance from different views. Therefore, we propose a novel method called Contrastive Label Enhancement (ConLE) which integrates features and logical labels into the unified projection space to generate high-level features by contrastive learning strategy. In this approach, features and logical labels belonging to the same sample are pulled closer, while those of different samples are projected farther away from each other in the projection space. Subsequently, we leverage the obtained high-level features to gain label distributions through a well-designed training strategy that considers the consistency of label attributes. Extensive experiments on LDL benchmark datasets demonstrate the effectiveness and superiority of our method.

Label distribution learning (LDL) is the state-of-the-art approach to dealing with a number of real-world applications, such as chronological age estimation from a face image, where there is an inherent similarity among adjacent age labels. LDL takes into account the semantic similarity by assigning a label distribution to each instance. The well-known Kullback-Leibler (KL) divergence is the widely used loss function for the LDL framework. However, the KL divergence does not fully and effectively capture the semantic similarity among age labels, thus leading to suboptimal performance. In this article, we propose a novel loss function based on the optimal transport theory for the LDL-based age estimation. A ground metric function plays an important role in the optimal transport formulation. It should be carefully determined based on the underlying geometric structure of the label space of the application in-hand. The label space in the age estimation problem has a specific geometric structure, that is, closer ages have more inherent semantic relationships. Inspired by this, we devise a novel ground metric function, which enables the loss function to increase the influence of highly correlated ages; thus exploiting the semantic similarity among ages more effectively than the existing loss functions. We then use the proposed loss function, namely, γ -Wasserstein loss, for training a deep neural network (DNN). This leads to a notoriously computationally expensive and nonconvex optimization problem. Following the standard methodology, we formulate the optimization function as a convex problem and then use an efficient iterative algorithm to update the parameters of the DNN. Extensive experiments in age estimation on different benchmark datasets validate the effectiveness of the proposed method, which consistently outperforms state-of-the-art approaches.

Feature selection based on label distribution and fuzzy mutual information

Clinical assessment of motor symptoms in Parkinson’s disease (PD) relies on rater-dependent scoring, which is subjective and may lead to substantial inter-rater variability. Conventional machine learning typically uses single-label targets and thus fails to capture the distributional nature of clinical ratings or the inherent ambiguity and continuity of disease progression. To address this issue, we propose PD-LDL, a multi-task label distribution learning (LDL) framework that jointly predicts MDS-UPDRS Part III scores and Hoehn & Yahr (H&Y) stages. PD-LDL employs a TabNet backbone for tabular feature selection and representation learning, and assigns Gaussian label distributions to model uncertainty in scoring and staging. However, directly learning large label distributions can increase output dimensionality and hamper optimization if label dependencies are ignored. To mitigate this, we introduce an adaptive label graph module inspired by UMAP, which constructs a fuzzy label affinity graph on the label distribution manifold and uses a graph-consistency loss to regularize model outputs according to learned structural correlations among labels. Experiments on the PPMI database demonstrate that PD-LDL consistently outperforms strong classical, deep tabular, and LDL baselines on both score prediction and staging tasks. The proposed framework not only improves predictive performance, but also more faithfully captures the continuity and ambiguity of PD-related labels, providing a interpretable tool for joint motor symptom scoring and staging in clinical practice.

Label distribution learning (LDL) to characterize the importance of different labels by label distribution has achieved good results in many application fields. LDL can learn more semantic information from the data than multi-label learning, however, most of the data in practical applications are single-label annotated or multi-label annotated, lacking the complete label distribution information suitable for label distribution learning. Thus, label enhancement (LE) is proposed to recover the label distributions from the logical labels. In this paper, we propose a new label enhancement method using inter-example correlation information that can automatically learn label correlations from data and jointly learn model and label correlations in a unified learning framework. Moreover, we also exploit the feature correlations constraining the model in the proposed method, which solves the problem that existing label enhancement algorithms cannot fully utilize the label information to improve the model performance. The experimental results on several real-world data sets validate the effectiveness of our method.KeywordsLabel enhancementLabel distribution learningLabel correlationFeature correlationLow rank