- Research Article
15
- 10.1016/j.neucom.2022.01.031
Multi-category classification with label noise by robust binary loss
- Jan 19, 2022
- Neurocomputing
- Defu Liu + 4 more +4
Multi-category classification with label noise by robust binary loss
Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis.
Multi-category classification with label noise by robust binary loss
Multi-category classification with label noise by robust binary loss
On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification
The development of accurate methods for multi-label classification (MLC) of remote sensing (RS) images is one of the most important research topics in RS. To address MLC problems, the use of deep neural networks that require a high number of reliable training images annotated by multiple land-cover class labels (multi-labels) has been found popular in RS. However, collecting such annotations is time-consuming and costly. A common procedure to obtain annotations at zero labeling cost is to rely on thematic products or crowdsourced labels. As a drawback, these procedures come with the risk of label noise that can distort the learning process of the MLC algorithms. In the literature, most label noise robust methods are designed for single-label classification (SLC) problems in computer vision (CV), where each image is annotated by a single label. Unlike SLC, label noise in MLC can be associated with: 1) subtractive label-noise (a land cover class label is not assigned to an image while that class is present in the image); 2) additive label-noise (a land cover class label is assigned to an image although that class is not present in the given image); and 3) mixed label-noise (a combination of both). In this paper, we investigate three different noise robust CV SLC methods and adapt them to be robust for multi-label noise scenarios in RS. During experiments, we study the effects of different types of multi-label noise and evaluate the adapted methods rigorously. To this end, we also introduce a synthetic multi-label noise injection strategy that is more adequate to simulate operational scenarios compared to the uniform label noise injection strategy, in which the labels of absent and present classes are flipped at uniform probability. Further, we study the relevance of different evaluation metrics in MLC problems under noisy multi-labels.
Read moreA survey of label-noise deep learning for medical image analysis
A survey of label-noise deep learning for medical image analysis
BadLabel: A Robust Perspective on Evaluating and Enhancing Label-Noise Learning.
Label-noise learning (LNL) aims to increase the model's generalization given training data with noisy labels. To facilitate practical LNL algorithms, researchers have proposed different label noise types, ranging from class-conditional to instance-dependent noises. In this paper, we introduce a novel label noise type called BadLabel, which can significantly degrade the performance of existing LNL algorithms by a large margin. BadLabel is crafted based on the label-flipping attack against standard classification, where specific samples are selected and their labels are flipped to other labels so that the loss values of clean and noisy labels become indistinguishable. To address the challenge posed by BadLabel, we further propose a robust LNL method that perturbs the labels in an adversarial manner at each epoch to make the loss values of clean and noisy labels again distinguishable. Once we select a small set of (mostly) clean labeled data, we can apply the techniques of semi-supervised learning to train the model accurately. Empirically, our experimental results demonstrate that existing LNL algorithms are vulnerable to the newly introduced BadLabel noise type, while our proposed robust LNL method can effectively improve the generalization performance of the model under various types of label noise. The new dataset of noisy labels and the source codes of robust LNL algorithms are available at https://github.com/zjfheart/BadLabels.
Read moreDistilling Effective Supervision From Severe Label Noise
Collecting large-scale data with clean labels for supervised training of neural networks is practically challenging. Although noisy labels are usually cheap to acquire, existing methods suffer a lot from label noise. This paper targets at the challenge of robust training at high label noise regimes. The key insight to achieve this goal is to wisely leverage a small trusted set to estimate exemplar weights and pseudo labels for noisy data in order to reuse them for supervised training. We present a holistic framework to train deep neural networks in a way that is highly invulnerable to label noise. Our method sets the new state of the art on various types of label noise and achieves excellent performance on large-scale datasets with real-world label noise. For instance, on CIFAR100 with a 40% uniform noise ratio and only 10 trusted labeled data per class, our method achieves 80.2% classification accuracy, where the error rate is only 1.4% higher than a neural network trained without label noise. Moreover, increasing the noise ratio to 80%, our method still maintains a high accuracy of 75.5%, compared to the previous best accuracy 48.2%.
Read moreRobust Binary Loss for Multi-Category Classification with Label Noise
Deep learning has achieved tremendous success in image classification. However, the corresponding performance leap relies heavily on large-scale accurate annotations, which are usually hard to collect in reality. It is essential to explore methods that can train deep models effectively under label noise. To address the problem, we propose to train deep models with robust binary loss functions. To be specific, we tackle the K-class classification task by using K binary classifiers. We can immediately use multi-category large margin classification approaches, e.g., Pairwise-Comparison (PC) or One-Versus-All (OVA), to jointly train the binary classifiers for multi-category classification. Our method can be robust to label noise if symmetric functions, e.g., the sigmoid loss or the ramp loss, are employed as the binary loss function in the framework of risk minimization. The learning theory reveals that our method can be inherently tolerant to label noise in multi-category classification tasks. Extensive experiments over different datasets with different types of label noise are conducted. The experimental results clearly confirm the effectiveness of our method.
Read moreDIER ‐Net: Debiased Learning With Medical Image Noisy Label by Intrinsic and Extrinsic Regularization
In medical image analysis, the presence of noisy labels and imbalanced data poses significant challenges to the performance of deep learning models, particularly in critical diagnostic tasks. To address this issue, we propose DIER‐Net, a learning with noisy label framework designed to handle noisy labels in imbalanced medical datasets. Our approach introduces a debiased sample selection technique that effectively filters out noisy labels while preserving important minority class samples. Additionally, we employ intrinsic and extrinsic regularization strategies to enhance the model's robustness by leveraging both clean and noisy data. Our method is evaluated on two widely used medical image datasets: the ISIC melanoma classification and Kaggle histopathologic lymph node classification. The experimental results demonstrate that DIER‐Net consistently outperforms existing state‐of‐the‐art methods, particularly in settings with high levels of label noise, offering a robust solution for real‐world clinical applications where noisy and imbalanced data are common. DIER‐Net provides an effective approach to enhance the reliability of AI systems in medical imaging, contributing to more accurate and trustworthy diagnostic outcomes.
Read moreChapter 14 - Deep learning in medical image analysis
Chapter 14 - Deep learning in medical image analysis
Ensemble Network-Based Distillation for Hyperspectral Image Classification in the Presence of Label Noise
Deep learning has made remarkable strides in hyperspectral image (HSI) classification, significantly improving classification performance. However, the challenge of obtaining accurately labeled training samples persists, primarily due to the subjectivity of human annotators and their limited domain knowledge. This often results in erroneous labels, commonly referred to as label noise. Such noisy labels can critically impair the performance of deep learning models, making it essential to address this issue. While previous studies focused on label noise filtering and label correction, these approaches often require estimating noise rates and may inadvertently propagate noisy labels to clean labels, especially in scenarios with high noise levels. In this study, we introduce an ensemble network-based distillation (END) method specifically designed to address the challenges posed by label noise in HSI classification. The core idea is to leverage multiple base neural networks to generate an estimated label distribution from the training data. This estimated distribution is then used alongside the ground-truth labels to train the target network effectively. Moreover, we propose a parameter-adaptive loss function that balances the impact of both the estimated and ground-truth label distributions during the training process. Our approach not only simplifies architectural requirements but also integrates seamlessly into existing deep learning frameworks. Comparative experiments on four hyperspectral datasets demonstrate the effectiveness of our method, highlighting its competitive performance in the presence of label noise.
Read moreGraph-Based Self-Learning for Robust Person Re-identification
Existing deep learning approaches for person re-identification (Re-ID) mostly rely on large-scale and well-annotated training data. However, human-annotated labels are prone to label noise in real-world applications. Previous person Re-ID works mainly focus on random label noise, which doesn’t properly reflect the characteristic of label noise in practical human-annotated process. In this work, we find the visual ambiguity noise is more common and reasonable noise assumption in annotation of person Re-ID. To handle the kind of noise, we propose a simple and effective robust person Re-ID framework, namely Graph-Based Self-Learning (GBSL), to iteratively learn discriminative representation and rectify noisy labels with limited annotated samples for each identity. Meanwhile, considering the practical annotation process in person Re-ID, we further extend the visual ambiguity noise assumption and propose a type of more practical label noise in person Re-ID, namely the tracklet-level label noise (TLN). Without modifying network architecture or loss function, our approach significantly improves the robustness against label noise of the Re-ID system. Our model obtains competitive performance with training data corrupted by various types of label noise and outperforms the existing methods for robust Re-ID on public benchmarks.
Read moreEFFECT OF LABEL NOISE IN SEMANTIC SEGMENTATION OF HIGH RESOLUTION AERIAL IMAGES AND HEIGHT DATA
Abstract. The performance of deep learning models in semantic segmentation is dependent on the availability of a large amount of labeled data. However, the influence of label noise, in the form of incorrect annotations, on the performance is significant and mostly ignored. This is a big concern in remote sensing applications, wherein acquired datasets are spatially limited, labeling is done by domain experts with possible sources of high inter-and intra-observer variability leading to erroneous predictions. In this paper, we first simulate the label noise while conducting experiments on two different datasets with very high-resolution aerial images, height data, and inaccurate labels, responsible for the training of deep learning models. We then focus on the effect of these noises on the model performance. Different classes respond differently to the label noise. The typical size of an object belonging to a class is a crucial factor regarding the class-specific performance of the model trained with erroneous labels. Errors caused by relative shifts of labels are the most influential label errors. The model is generally more tolerant of the random label noise than other label errors. It has been observed that the accuracy gets reduced by at least 3% while 5% of label pixels are erroneous. In this regard, our study provides a new perspective of evaluating and quantifying the propagation of label noise in the model performance that is indeed important for adopting reliable semantic segmentation practices.
Read moreRobust Medical Image Classification From Noisy Labeled Data With Global and Local Representation Guided Co-Training.
Deep neural networks have achieved remarkable success in a wide variety of natural image and medical image computing tasks. However, these achievements indispensably rely on accurately annotated training data. If encountering some noisy-labeled images, the network training procedure would suffer from difficulties, leading to a sub-optimal classifier. This problem is even more severe in the medical image analysis field, as the annotation quality of medical images heavily relies on the expertise and experience of annotators. In this paper, we propose a novel collaborative training paradigm with global and local representation learning for robust medical image classification from noisy-labeled data to combat the lack of high quality annotated medical data. Specifically, we employ the self-ensemble model with a noisy label filter to efficiently select the clean and noisy samples. Then, the clean samples are trained by a collaborative training strategy to eliminate the disturbance from imperfect labeled samples. Notably, we further design a novel global and local representation learning scheme to implicitly regularize the networks to utilize noisy samples in a self-supervised manner. We evaluated our proposed robust learning strategy on four public medical image classification datasets with three types of label noise, i.e., random noise, computer-generated label noise, and inter-observer variability noise. Our method outperforms other learning from noisy label methods and we also conducted extensive experiments to analyze each component of our method.
Read moreA comprehensive survey on deep active learning in medical image analysis
A comprehensive survey on deep active learning in medical image analysis
Recent advances and clinical applications of deep learning in medical image analysis.
Recent advances and clinical applications of deep learning in medical image analysis.
Intelligent agent for hyperspectral image classification with noisy labels: a deep reinforcement learning framework
Label accuracy plays an important role in supervised hyperspectral image classification problems. However, labelling can be prone to various types of noise, mainly including human errors and incorrect pseudo-label predictions. Learning from datasets with noisy labels is a critical challenge in practice. To address this issue, we first propose a deep reinforcement learning method named HyperRefine to tackle the label noise problem for hyperspectral image classification. The method utilizes an agent to learn the accuracy of each sample label and determine its suitability to be used in training the classifier model. The agent is trained using a reinforcement signal based on the reward obtained from a small validation set that reflects performance on the target task, where the validation set is generated from the training set by a momentum-based recruiter. Compared to complex hand-crafted methods of label noise cleaning, HyperRefine is easy to implement, yet highly effective. The robustness of the model to different types of label noises is significantly improved by further combining a novel label refinement approach. Experiments on artificially added label noise and pseudo-label noise hyperspectral datasets show that the proposed method can achieve promising performance.
Read more