• Home
  • Search
  • Learning Student Network Under Universal Label Noise.
  • Cite Icon9
  • https://doi.org/10.1109/tip.2024.3430539Copy DOI Icon

Learning Student Network Under Universal Label Noise.

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Data-free knowledge distillation aims to learn a small student network from a large pre-trained teacher network without the aid of original training data. Recent works propose to gather alternative data from the Internet for training student network. In a more realistic scenario, the data on the Internet contains two types of label noise, namely: 1) closed-set label noise, where some examples belong to the known categories but are mislabeled; and 2) open-set label noise, where the true labels of some mislabeled examples are outside the known categories. However, the latter is largely ignored by existing works, leading to limited student network performance. Therefore, this paper proposes a novel data-free knowledge distillation paradigm by utilizing a webly-collected dataset under universal label noise, which means both closed-set and open-set label noise should be tackled. Specifically, we first split the collected noisy dataset into clean set, closed noisy set, and open noisy set based on the prediction uncertainty of various data types. For the closed-set noisy examples, their labels are refined by teacher network. Meanwhile, a noise-robust hybrid contrastive learning is performed on the clean set and refined closed noisy set to encourage student network to learn the categorical and instance knowledge inherited by teacher network. For the open-set noisy examples unexplored by previous work, we regard them as unlabeled and conduct self-supervised learning on them to enrich the supervision signal for student network. Intensive experimental results on image classification tasks demonstrate that our approach can achieve superior performance to state-of-the-art data-free knowledge distillation methods.

Similar Papers
  • Research Article
  • Citations702

Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis.

  • Jun 20, 2020
  • Medical image analysis
  • Davood Karimi +3
  • Research Article
  • Citations46

On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification

  • Jan 01, 2022
  • IEEE Transactions on Geoscience and Remote Sensing
  • Tom Burgert +2
  • Conference Article
  • Citations3

Importance Reweighting for Biquality Learning

  • Jul 18, 2021
  • Pierre Nodet +4
  • Conference Article
  • Citations173

Distilling Effective Supervision From Severe Label Noise

  • Jun 01, 2020
  • Zizhao Zhang +4
  • Conference Article
  • Citations7

Graph-Based Self-Learning for Robust Person Re-identification

  • Jan 01, 2023
  • Yuqiao Xian +4
  • Book Chapter
  • Citations8

Effect of Label Noise on Robustness of Deep Neural Network Object Detectors

  • Jan 01, 2021
  • Bishwo Adhikari +4
  • Conference Article
  • Citations1

An OpenStreetMap-Based Dataset of Building Footprints for Analysing Different Types of Label Noise

  • Jul 11, 2021
  • Jonas Gutter +2
  • Research Article
  • Citations29

BadLabel: A Robust Perspective on Evaluating and Enhancing Label-Noise Learning.

  • Jun 01, 2024
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Jingfeng Zhang +6
  • Research Article
  • Citations11

Unlocking the Power of Open Set: A New Perspective for Open-Set Noisy Label Learning

  • Mar 24, 2024
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Wenhai Wan +5
  • Conference Article
  • Citations7

Robust Binary Loss for Multi-Category Classification with Label Noise

  • Jun 06, 2021
  • Defu Liu +4
  • Conference Article
  • Citations2

Making Deep Neural Networks Robust to Label Noise: A Reweighting Loss and Data Filtration

  • Jul 01, 2019
  • Zhengwen Zhang +3
  • Research Article
  • Citations5

Label noise detection under the noise at random model with ensemble filters

  • Sep 05, 2022
  • Intelligent Data Analysis
  • Kecia G Moura +2
  • Conference Article
  • Citations1

Ensemble Methods for Label Noise Detection Under the Noisy at Random Model

  • Oct 01, 2018
  • Kecia G De Moura +2
  • Research Article
  • Citations15

Multi-category classification with label noise by robust binary loss

  • Jan 19, 2022
  • Neurocomputing
  • Defu Liu +4
  • Research Article
  • Citations46

Robust Distance Metric Learning via Bayesian Inference.

  • Dec 11, 2017
  • IEEE Transactions on Image Processing
  • Dong Wang +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.