• Home
  • Search
  • Self-Supervised Learning of Pretext-Invariant Representations
  • Cite Icon1518
  • https://doi.org/10.1109/cvpr42600.2020.00674Copy DOI Icon

Self-Supervised Learning of Pretext-Invariant Representations

  • Jun 1, 2020
  • Ishan Misra +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The goal of self-supervised learning from images is to construct image representations that are semantically meaningful via pretext tasks that do not require semantic annotations. Many pretext tasks lead to representations that are covariant with image transformations. We argue that, instead, semantic representations ought to be invariant under such transformations. Specifically, we develop Pretext-Invariant Representation Learning (PIRL, pronounced as `pearl') that learns invariant representations based on pretext tasks. We use PIRL with a commonly used pretext task that involves solving jigsaw puzzles. We find that PIRL substantially improves the semantic quality of the learned image representations. Our approach sets a new state-of-the-art in self-supervised learning from images on several popular benchmarks for self-supervised learning. Despite being unsupervised, PIRL outperforms supervised pre-training in learning image representations for object detection. Altogether, our results demonstrate the potential of self-supervised representations with good invariance properties.

Similar Papers
  • Research Article
  • Citations8

Self-supervised learning for heterogeneous graph via structure information based on metapath

  • May 12, 2023
  • Applied Soft Computing
  • Shuai Ma +2
  • Conference Article
  • Citations2

A Novel Multi-Task Self-Supervised Representation Learning Paradigm

  • May 28, 2021
  • Control theory & applications
  • Yinggang Li +5
  • Research Article
  • Citations45

Volumetric white matter tract segmentation with nested self-supervised learning using sequential pretext tasks.

  • Apr 30, 2021
  • Medical Image Analysis
  • Qi Lu +2
  • Book Chapter
  • Citations7

CaSS: A Channel-Aware Self-supervised Representation Learning Framework for Multivariate Time Series Classification

  • Jan 01, 2022
  • Yijiang Chen +4
  • Research Article
  • Citations16

Pretext Tasks Selection for Multitask Self-Supervised Audio Representation Learning

  • Oct 01, 2022
  • IEEE Journal of Selected Topics in Signal Processing
  • Salah Zaiem +3
  • Research Article
  • Citations20

Self-supervised visual learning in the low-data regime: A comparative evaluation

  • Mar 01, 2025
  • Neurocomputing
  • Sotirios Konstantakos +6
  • Research Article

Self-supervised learning for stroke lesion segmentation on CT: a new pretext task for neuroimaging.

  • Jul 14, 2025
  • Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
  • Juliette Moreau +6
  • PDF
  • Conference Article
  • Citations3

Organoids Segmentation using Self-Supervised Learning: How Complex Should the Pretext Task Be?

  • Nov 09, 2023
  • Asmaa Haja +2
  • PDF
  • Research Article
  • Citations16

A Generic Self-Supervised Learning (SSL) Framework for Representation Learning from Spectral–Spatial Features of Unlabeled Remote Sensing Imagery

  • Nov 03, 2023
  • Remote Sensing
  • Xin Zhang +1
  • Conference Article
  • Citations64

Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting

  • Jun 01, 2021
  • Ayan Kumar Bhunia +5
  • Research Article
  • Citations6

Self-Supervised Video Representation Learning by Serial Restoration With Elastic Complexity

  • Jan 01, 2024
  • IEEE Transactions on Multimedia
  • Ziyu Chen +2
  • Conference Article
  • Citations4

Conditional Independence for Pretext Task Selection in Self-Supervised Speech Representation Learning

  • Aug 30, 2021
  • Salah Zaiem +2
  • Conference Article
  • Citations4

Self-supervised Learning Based on Max-tree Representation for Medical Image Segmentation

  • Jul 18, 2022
  • Qian Tang +2
  • Conference Article

Self-supervised multi-task learning for semantic segmentation of urban scenes

  • Sep 12, 2021
  • Jonathan Gonzalez-Santiago +2
  • Research Article
  • Citations3

Evolutionary Augmentation Policy Optimization for Self-Supervised Learning

  • Jan 01, 2023
  • Advances in Artificial Intelligence and Machine Learning
  • Noah Barrett +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.