• Home
  • Search
  • DATA: Domain-Aware and Task-Aware Self-supervised Learning
  • Open Access IconOpen Access
  • Cite Icon9
  • https://doi.org/10.1109/cvpr52688.2022.00961Copy DOI Icon

DATA: Domain-Aware and Task-Aware Self-supervised Learning

  • Jun 1, 2022
  • Qing Chang +6 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The paradigm of training models on massive data without label through self-supervised learning (SSL) and fine-tuning on many downstream tasks has become a trend recently. However, due to the high training costs and the un-consciousness of downstream usages, most self-supervised learning methods lack the capability to correspond to the diversities of downstream scenarios, as there are various data domains, different vision tasks and latency constraints on models. Neural architecture search (NAS) is one universally acknowledged fashion to conquer the issues above, but applying NAS on SSL seems impossible as there is no label or metric provided for judging model selection. In this paper, we present DATA, a simple yet effective NAS approach specialized for SSL that provides Domain-Aware and Task-Aware pre-training. Specifically, we (i) train a supernet which could be deemed as a set of millions of networks covering a wide range of model scales without any label, (ii) propose a flexible searching mechanism compatible with SSL that enables finding networks of different computation costs, for various downstream vision tasks and data domains without explicit metric provided. Instantiated With MoCo v2, our method achieves promising results across a wide range of computation costs on down-stream tasks, including image classification, object detection and semantic segmentation. DATA is orthogonal to most existing SSL methods and endows them the ability of customization on downstream needs. Extensive experiments on other SSL methods demonstrate the generalizability of the proposed method. Code is released at https://github.com/GAIA-vision/GAIA-ssl.

Similar Papers
  • Research Article
  • Citations14

Benchmarking Self-Supervised Contrastive Learning Methods for Image-Based Plant Phenotyping

  • Jan 01, 2023
  • Plant Phenomics
  • Franklin C Ogidi +2
  • Conference Article
  • Citations18

Self-supervised Visual Attribute Learning for Fashion Compatibility

  • Oct 01, 2021
  • Donghyun Kim +5
  • Conference Article
  • Citations2

A Novel Multi-Task Self-Supervised Representation Learning Paradigm

  • May 28, 2021
  • Control theory & applications
  • Yinggang Li +5
  • Research Article
  • Citations20

Self-supervised visual learning in the low-data regime: A comparative evaluation

  • Mar 01, 2025
  • Neurocomputing
  • Sotirios Konstantakos +6
  • Conference Article
  • Citations10

Self-supervised Learning with Local Contrastive Loss for Detection and Semantic Segmentation

  • Jan 01, 2023
  • Ashraful Islam +5
  • PDF
  • Research Article
  • Citations1

CDEST: Class Distinguishability-Enhanced Self-Training Method for Adopting Pre-Trained Models to Downstream Remote Sensing Image Semantic Segmentation

  • Apr 06, 2024
  • Remote Sensing
  • Ming Zhang +7
  • Conference Article
  • Citations12

Conformer-Based Self-Supervised Learning For Non-Speech Audio Tasks

  • May 23, 2022
  • Sangeeta Srivastava +6
  • Research Article
  • Citations7

Randomly shuffled convolution for self-supervised representation learning

  • Nov 14, 2022
  • Information Sciences
  • Youngjin Oh +3
  • Research Article
  • Citations6

Deeply Supervised Block-Wise Neural Architecture Search.

  • Feb 01, 2025
  • IEEE transactions on neural networks and learning systems
  • An Yang +3
  • Research Article
  • Citations4

WISE: whole-scenario embryo identification using self-supervised learning encoder in IVF.

  • Mar 12, 2024
  • Journal of assisted reproduction and genetics
  • Mark Liu +5
  • Research Article
  • Citations94

CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding

  • Jan 01, 2023
  • IEEE Transactions on Geoscience and Remote Sensing
  • Dilxat Muhtar +4
  • Research Article
  • Citations48

CS-CO: A Hybrid Self-Supervised Visual Representation Learning Method for H&E-stained Histopathological Images.

  • Oct 01, 2022
  • Medical Image Analysis
  • Pengshuai Yang +6
  • PDF
  • Research Article
  • Citations41

GAN-Based Image Colorization for Self-Supervised Visual Feature Learning.

  • Feb 18, 2022
  • Sensors (Basel, Switzerland)
  • Sandra Treneska +4
  • Research Article
  • Citations12

DenseCL: A simple framework for self-supervised dense visual pre-training

  • Sep 24, 2022
  • Visual Informatics
  • Xinlong Wang +3
  • Research Article
  • Citations5

Self-Supervised Learning with Atom Replacement for Catalyst Energy Prediction by Graph Neural Networks

  • Jan 01, 2023
  • Procedia Computer Science
  • Yasufumi Sakai +8
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.