• Home
  • Search
  • Self-supervised Learning: Generative or Contrastive
  • Open Access IconOpen Access
  • Cite Icon1808
  • https://doi.org/10.1109/tkde.2021.3090866Copy DOI Icon

Self-supervised Learning: Generative or Contrastive

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Deep supervised learning has achieved great success in the last decade. However, its deficiencies of dependence on manual labels and vulnerability to attacks have driven people to explore a better solution. As an alternative, self-supervised learning attracts many researchers for its soaring performance on representation learning in the last several years. Self-supervised representation learning leverages input data itself as supervision and benefits almost all types of downstream tasks. In this survey, we take a look into new self-supervised learning methods for representation in computer vision, natural language processing, and graph learning. We comprehensively review the existing empirical methods and summarize them into three main categories according to their objectives: generative, contrastive, and generative-contrastive (adversarial). We further investigate related theoretical analysis work to provide deeper thoughts on how self-supervised learning works. Finally, we briefly discuss open problems and future directions for self-supervised learning. An outline slide for the survey is provided.

Similar Papers
  • Conference Article
  • Citations2

A Novel Multi-Task Self-Supervised Representation Learning Paradigm

  • May 28, 2021
  • Control theory & applications
  • Yinggang Li +5
  • Research Article

Self-Supervised Deep Learning Models for Low-Resource NLP Applications

  • Feb 26, 2026
  • International Journal of Research and Review in Applied Science, Humanities, and Technology
  • Pardeep Kaur
  • Research Article
  • Citations114

Graph Barlow Twins: A self-supervised representation learning framework for graphs

  • Aug 17, 2022
  • Knowledge-Based Systems
  • Piotr Bielak +2
  • Research Article

Abstract 5059: Self-supervised representation learning of somatic mutational data

  • Apr 21, 2025
  • Cancer Research
  • Marzieh Haghighi +2
  • Conference Article
  • Citations58

LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speech

  • Aug 30, 2021
  • Solène Evain +17
  • Conference Article
  • Citations16

A Novel Solution for EEG-based Emotion Recognition

  • Oct 13, 2021
  • Zhuofan Xie +2
  • PDF
  • Book Chapter
  • Citations13

Self-Supervised Contrastive Representation Learning in Computer Vision

  • Dec 21, 2022
  • Yalin Bastanlar +1
  • Book Chapter
  • Citations7

CaSS: A Channel-Aware Self-supervised Representation Learning Framework for Multivariate Time Series Classification

  • Jan 01, 2022
  • Yijiang Chen +4
  • Research Article
  • Citations14

Benchmarking Self-Supervised Contrastive Learning Methods for Image-Based Plant Phenotyping

  • Jan 01, 2023
  • Plant Phenomics
  • Franklin C Ogidi +2
  • Preprint Article

Self-Supervised Learning Principles Challenges and Emerging Directions

  • Feb 24, 2025
  • Preprints.org
  • Jenifer Nadine
  • PDF
  • Research Article
  • Citations4

Self-supervised maize kernel classification and segmentation for embryo identification

  • Apr 14, 2023
  • Frontiers in Plant Science
  • David Dong +6
  • Research Article
  • Citations20

Self-supervised visual learning in the low-data regime: A comparative evaluation

  • Mar 01, 2025
  • Neurocomputing
  • Sotirios Konstantakos +6
  • Research Article
  • Citations1

Can Generative Models Improve Self-Supervised Representation Learning?

  • Apr 11, 2025
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Sana Ayromlou +3
  • Research Article
  • Citations2

Research on Self-Supervised Comparative Learning for Computer Vision

  • Aug 17, 2021
  • Journal of Electronic Research and Application
  • Yuanyuan Liu +1
  • Research Article
  • Citations7

MLVICX: Multi-Level Variance-Covariance Exploration for Chest X-Ray Self-Supervised Representation Learning.

  • Dec 01, 2024
  • IEEE journal of biomedical and health informatics
  • Azad Singh +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.