• Home
  • Search
  • Divide and Contrast: Self-supervised Learning from Uncurated Data
  • Open Access IconOpen Access
  • Cite Icon51
  • https://doi.org/10.1109/iccv48922.2021.00991Copy DOI Icon

Divide and Contrast: Self-supervised Learning from Uncurated Data

  • Oct 1, 2021
  • Yonglong Tian +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Self-supervised learning holds promise in leveraging large amounts of unlabeled data, however much of its progress has thus far been limited to highly curated pre-training data such as ImageNet. We explore the effects of contrastive learning from larger, less-curated image datasets such as YFCC, and find there is indeed a large difference in the resulting representation quality. We hypothesize that this curation gap is due to a shift in the distribution of image classes—which is more diverse and heavy-tailed—resulting in less relevant negative samples to learn from. We test this hypothesis with a new approach, Divide and Contrast (DnC), which alternates between contrastive learning and clustering-based hard negative mining. When pretrained on less curated datasets, DnC greatly improves the performance of self-supervised learning on downstream tasks, while remaining competitive with the current state-of-the-art on curated datasets.

Similar Papers
  • PDF
  • Research Article
  • Citations1

CDEST: Class Distinguishability-Enhanced Self-Training Method for Adopting Pre-Trained Models to Downstream Remote Sensing Image Semantic Segmentation

  • Apr 06, 2024
  • Remote Sensing
  • Ming Zhang +7
  • Preprint Article

Generalist Geospatial Foundation Model PhilEO on Satellite Sentinel-2 MajorTOM Multi-Spectral Data

  • Mar 15, 2025
  • Nikolaos Dionelis +9
  • Conference Article
  • Citations1

Self-Supervised Contrastive Learning on Cross-Augmented Samples for SAR Target Recognition

  • May 01, 2023
  • Xiaoyu Liu +6
  • Research Article
  • Citations2

FGSS: Federated global self-supervised framework for large-scale unlabeled data

  • May 24, 2023
  • Applied Soft Computing
  • Chen Zhang +4
  • PDF
  • Research Article
  • Citations16

A Generic Self-Supervised Learning (SSL) Framework for Representation Learning from Spectral–Spatial Features of Unlabeled Remote Sensing Imagery

  • Nov 03, 2023
  • Remote Sensing
  • Xin Zhang +1
  • Research Article
  • Citations5

Contrastive Self-supervised EEG Representation Learning for Emotion Classification.

  • Jul 15, 2024
  • Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
  • Keya Hu +5
  • Conference Article
  • Citations2

A Novel Multi-Task Self-Supervised Representation Learning Paradigm

  • May 28, 2021
  • Control theory & applications
  • Yinggang Li +5
  • PDF
  • Research Article
  • Citations4

Deep learning based on self-supervised pre-training: Application on sandstone content prediction

  • Jan 10, 2023
  • Frontiers in Earth Science
  • Chong Ming Wang +5
  • PDF
  • Research Article
  • Citations20

Self-supervised learning for remote sensing scene classification under the few shot scenario

  • Jan 09, 2023
  • Scientific Reports
  • Najd Alosaimi +4
  • Research Article
  • Citations390

A comparison review of transfer learning and self-supervised learning: Definitions, applications, advantages and limitations

  • Dec 02, 2023
  • Expert Systems with Applications
  • Zehui Zhao +4
  • Research Article

Dynamic Entity-Masked Graph Diffusion Model for Histopathology Image Representation Learning

  • Apr 11, 2025
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Zhenfeng Zhuang +6
  • Conference Article
  • Citations4

Self-supervised Learning Based on Max-tree Representation for Medical Image Segmentation

  • Jul 18, 2022
  • Qian Tang +2
  • Research Article
  • Citations43

Steel surface defect detection based on self-supervised contrastive representation learning with matching metric

  • Jun 30, 2023
  • Applied Soft Computing
  • Xuejin Hu +5
  • PDF
  • Research Article
  • Citations5

GFCNet: Contrastive Learning Network with Geography Feature Space Joint Negative Sample Correction for Land Cover Classification

  • Oct 21, 2023
  • Remote Sensing
  • Zhaoyang Zhang +4
  • Research Article
  • Citations20

Self-supervised visual learning in the low-data regime: A comparative evaluation

  • Mar 01, 2025
  • Neurocomputing
  • Sotirios Konstantakos +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.