• Home
  • Search
  • Conformer-Based Self-Supervised Learning For Non-Speech Audio Tasks
  • Open Access IconOpen Access
  • Cite Icon12
  • https://doi.org/10.1109/icassp43922.2022.9746490Copy DOI Icon

Conformer-Based Self-Supervised Learning For Non-Speech Audio Tasks

  • May 23, 2022
  • Sangeeta Srivastava +6 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Representation learning from unlabeled data has been of major interest in artificial intelligence research. While self-supervised speech representation learning has been popular in the speech research community, very few works have comprehensively analyzed audio representation learning for non-speech audio tasks. In this paper, we propose a self-supervised audio representation learning method and apply it to a variety of downstream non-speech audio tasks. We combine the well-known wav2vec 2.0 framework, which has shown success in self-supervised learning for speech tasks, with parameter-efficient conformer architectures. Our self-supervised pre-training can reduce the need for labeled data by two-thirds. On the AudioSet benchmark, we achieve a mean average precision (mAP) score of 0.415, which is a new state-of-the-art on this dataset through audio-only self-supervised learning. Our fine-tuned conformers also surpass or match the performance of previous systems pre-trained in a supervised way on several downstream tasks. We further discuss the important design considerations for both pre-training and fine-tuning.

Similar Papers
  • Research Article
  • Citations14

Benchmarking Self-Supervised Contrastive Learning Methods for Image-Based Plant Phenotyping

  • Jan 01, 2023
  • Plant Phenomics
  • Franklin C Ogidi +2
  • Conference Article
  • Citations2

A Novel Multi-Task Self-Supervised Representation Learning Paradigm

  • May 28, 2021
  • Control theory & applications
  • Yinggang Li +5
  • Research Article
  • Citations20

Self-supervised visual learning in the low-data regime: A comparative evaluation

  • Mar 01, 2025
  • Neurocomputing
  • Sotirios Konstantakos +6
  • Conference Article
  • Citations18

Self-supervised Visual Attribute Learning for Fashion Compatibility

  • Oct 01, 2021
  • Donghyun Kim +5
  • Research Article
  • Citations7

MLVICX: Multi-Level Variance-Covariance Exploration for Chest X-Ray Self-Supervised Representation Learning.

  • Dec 01, 2024
  • IEEE journal of biomedical and health informatics
  • Azad Singh +2
  • Research Article

Enhancing Slice-Wise Brain MRI Tasks using Self-Supervised and Auxiliary Learning.

  • Jan 01, 2025
  • IEEE journal of biomedical and health informatics
  • Giovanni Castiglioni +4
  • Research Article
  • Citations16

Self-supervised in-domain representation learning for remote sensing image scene classification

  • Sep 14, 2024
  • Heliyon
  • Ali Ghanbarzadeh +1
  • Research Article
  • Citations2

Spatiotemporal self-supervised predictive learning for atmospheric variable prediction via multi-group multi-attention

  • Jun 13, 2024
  • Knowledge-Based Systems
  • Zhensheng Shi +2
  • Research Article

Abstract 5059: Self-supervised representation learning of somatic mutational data

  • Apr 21, 2025
  • Cancer Research
  • Marzieh Haghighi +2
  • Research Article
  • Citations48

CS-CO: A Hybrid Self-Supervised Visual Representation Learning Method for H&E-stained Histopathological Images.

  • Oct 01, 2022
  • Medical Image Analysis
  • Pengshuai Yang +6
  • Conference Article
  • Citations9

DATA: Domain-Aware and Task-Aware Self-supervised Learning

  • Jun 01, 2022
  • Qing Chang +6
  • Research Article
  • Citations5

Self-Supervised Learning with Atom Replacement for Catalyst Energy Prediction by Graph Neural Networks

  • Jan 01, 2023
  • Procedia Computer Science
  • Yasufumi Sakai +8
  • Research Article
  • Citations2

Research on Self-Supervised Comparative Learning for Computer Vision

  • Aug 17, 2021
  • Journal of Electronic Research and Application
  • Yuanyuan Liu +1
  • Research Article
  • Citations1

Can Generative Models Improve Self-Supervised Representation Learning?

  • Apr 11, 2025
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Sana Ayromlou +3
  • Research Article
  • Citations11

Joint Self-Supervised Image-Volume Representation Learning with Intra-inter Contrastive Clustering

  • Jun 26, 2023
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Duy M H Nguyen +9
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.