• Home
  • Search
  • Self-Supervised Video Representation Learning by Serial Restoration With Elastic Complexity
  • Cite Icon5
  • https://doi.org/10.1109/tmm.2023.3293727Copy DOI Icon

Self-Supervised Video Representation Learning by Serial Restoration With Elastic Complexity

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Self-supervised video representation learning leaves out heavy manual annotation by automatically excavating supervisory signals. Although contrastive learning based approaches exhibit superior performances, pretext task based approaches still deserve further study. This is because the pretext tasks exploit the nature of data and encourage feature extractors to learn spatiotemporal logic by discovering dependencies among video clips or cubes, without manual engineering on data augmentations or manual construction of contrastive pairs. To utilize chronological property more effectively and efficiently, this work proposes a novel pretext task, named serial restoration of shuffled clips (SRSC), disentangled by an elaborately designed task network composed of an order-aware encoder and a serial restoration decoder. In contrast to other order based pretext tasks that formulate clip order recognition as a one-step classification problem, the proposed SRSC task restores shuffled clips into the right order in multiple steps. Owing to the excellent elasticity of SRSC, a novel taxonomy of curriculum learning is further proposed to equip SRSC with different pre-training strategies. According to the factors that affect the complexity of solving the SRSC task, the proposed curriculum learning strategies can be categorized into task based, model based and data based. Extensive experiments are conducted on the subdivided strategies to explore their effectiveness and noteworthy laws. Compared with existing approaches, this work demonstrates that the proposed approach achieves state-of-the-art performances in pretext task based self-supervised video representation learning and a majority of the proposed strategies further boost the performance of downstream tasks. For the first time, the features pre-trained by the pretext tasks are applied to video captioning by feature-level early fusion, and enhance the input of existing approaches as a lightweight plugin.

Similar Papers
  • Conference Article
  • Citations57

Self-supervised Video Representation Learning by Context and Motion Decoupling

  • Jun 01, 2021
  • Lianghua Huang +5
  • PDF
  • Research Article
  • Citations1

PatchLoc: Embedded Patch Localization Pretext Task for Tumor Segmentation in Medical Images

  • Jan 01, 2024
  • IEEE Access
  • Ramchandra Cheke +3
  • PDF
  • Research Article
  • Citations9

A Contrastive Predictive Coding-Based Classification Framework for Healthcare Sensor Data.

  • Mar 15, 2022
  • Journal of healthcare engineering
  • Chaoxu Ren +2
  • Research Article
  • Citations7

GenSelfDiff-HIS: Generative Self-Supervision Using Diffusion for Histopathological Image Segmentation.

  • Feb 01, 2025
  • IEEE transactions on medical imaging
  • Vishnuvardhan Purma +4
  • Research Article
  • Citations4

An exploratory study of self-supervised pre-training on partially supervised multi-label classification on chest X-ray images

  • Jun 14, 2024
  • Applied Soft Computing
  • Nanqing Dong +3
  • Research Article
  • Citations1

Exploring complementary information of self‐supervised pretext tasks for unsupervised video pre‐training

  • Nov 28, 2021
  • IET Computer Vision
  • Wei Zhou +3
  • Book Chapter
  • Citations11

Tailoring Self-Supervision for Supervised Learning

  • Jan 01, 2022
  • Wonjun Moon +2
  • Book Chapter
  • Citations15

Concurrent Subsidiary Supervision for Unsupervised Source-Free Domain Adaptation

  • Jan 01, 2022
  • Jogendra Nath Kundu +5
  • Research Article
  • Citations350

A comparison review of transfer learning and self-supervised learning: Definitions, applications, advantages and limitations

  • Dec 02, 2023
  • Expert Systems with Applications
  • Zehui Zhao +4
  • Research Article
  • Citations14

SMG: self-supervised masked graph learning for cancer gene identification.

  • Sep 22, 2023
  • Briefings in Bioinformatics
  • Yan Cui +10
  • Research Article
  • Citations22

Self-supervised learning framework application for medical image analysis: a review and summary

  • Oct 27, 2024
  • BioMedical Engineering OnLine
  • Xiangrui Zeng +2
  • Conference Article
  • Citations6

Self-Supervised Learning of Audio Representations using Angular Contrastive Loss

  • Jun 04, 2023
  • Shanshan Wang +2
  • Conference Article

A Self-Supervised Learning Framework for Low-Resource Computer Vision Applications

  • Jan 27, 2026
  • Rupam Sah +5
  • Research Article

A transformer fault diagnosis method for small-sample imbalanced data based on self-supervised representation learning

  • Jul 11, 2025
  • Intelligent Decision Technologies
  • Meng Guo +6
  • Research Article
  • Citations42

PathoDuet: Foundation models for pathological slide analysis of H&E and IHC stains

  • Jul 31, 2024
  • Medical Image Analysis
  • Shengyi Hua +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.