• Home
  • Search
  • Revamped Knowledge Distillation for Sound Classification
  • Cite Icon3
  • https://doi.org/10.1109/ijcnn55064.2022.9892474Copy DOI Icon

Revamped Knowledge Distillation for Sound Classification

  • Jul 18, 2022
  • Achyut Mani Tripathi +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

This paper presents a novel knowledge distillation technique that inherits knowledge from multiple deep Environment Sound Classification (ESC) models trained on spectrogram features created by dividing the spectrogram into multiple subband spectrogram. The deep models trained on sub-band spectrograms prevent information loss while performing knowledge distillation from a teacher model to a student model receiving the full spectrogram as an input. The student models' performance is evaluated on two benchmark sound datasets, viz. the ESC-10 and Audio MNIST datasets. The impact of teacher models trained with different number of sub-band features and four ensemble techniques has been investigated thoroughly to enhance the final accuracy of the student model supervised by the proposed knowledge distillation framework. Experiments and results shows that the accuracy of the student model is comparable and competitive to state-of-the-art methods for sound classification. Moreover, the student model trained on the Audio MNIST dataset attains an hitherto unpublished accuracy of 98.25%, a new benchmark for the Audio MNIST dataset. Additionally, Grad- CAM visualization of the spectrogram features is generated to identify the spectrogram's relevant regions and understand why the model classifies a signal into a specific class.

Similar Papers
  • Research Article
  • Citations44

Environment sound classification using an attention-based residual neural network

  • Jun 17, 2021
  • Neurocomputing
  • Achyut Mani Tripathi +1
  • PDF
  • Research Article
  • Citations15

Mitigating carbon footprint for knowledge distillation based deep learning model compression.

  • May 15, 2023
  • PLOS ONE
  • Kazi Rafat +7
  • Conference Article
  • Citations19

Sub-spectrogram Segmentation for Environmental Sound Classification via Convolutional Recurrent Neural Network and Score Level Fusion

  • Oct 01, 2019
  • Tianhao Qiao +4
  • Conference Article
  • Citations5

Research on Environmental Sound Classification Algorithm Based on Multi-feature Fusion

  • Dec 11, 2020
  • Ruixue Li +4
  • Conference Article
  • Citations1

Lightweight Network for Vietnamese Landmark Recognition based on Knowledge Distillation

  • Nov 26, 2020
  • Viet T Tran +3
  • Conference Article
  • Citations3

Stage-by-stage knowledge distillation

  • Jan 27, 2021
  • Mengya Gao +3
  • Research Article
  • Citations2

De-biased knowledge distillation framework based on knowledge infusion and label de-biasing techniques

  • Aug 16, 2024
  • Journal of Electronic Science and Technology
  • Yan Li +3
  • PDF
  • Conference Article
  • Citations87

Urban Sound Classification using Long Short-Term Memory Neural Network

  • Sep 26, 2019
  • Iurii Lezhenin +2
  • Research Article
  • Citations3

Equity-enhanced glaucoma progression prediction from OCT with knowledge distillation.

  • Jul 24, 2025
  • NPJ digital medicine
  • Sulaiman O Afolabi +5
  • Research Article
  • Citations8

Cosine similarity-guided knowledge distillation for robust object detectors

  • Aug 14, 2024
  • Scientific Reports
  • Sangwoo Park +2
  • Research Article

MCW-KD: Multi-Cost Wasserstein Knowledge Distillation for Large Language Models

  • Mar 14, 2026
  • Hoang Tran Vuong +3
  • Research Article

AMAKD: Adversarial multi-modal attention-based knowledge distillation for robust behaviour anomaly detection in real-world environments

  • Apr 01, 2026
  • Alexandria Engineering Journal
  • Maram Fahaad Almufareh
  • Conference Article
  • Citations1

Research of Weibo Text Classification based on Knowledge Distillation and Joint Model

  • Mar 12, 2021
  • Zhengwu Yuan +1
  • Conference Article
  • Citations5

Subclass Knowledge Distillation with Known Subclass Labels

  • Jun 26, 2022
  • Ahmad Sajedi +2
  • Research Article
  • Citations4

HyperKD: Lifelong Hyperspectral Image Classification With Cross-Spectral–Spatial Knowledge Distillation

  • Jan 01, 2025
  • IEEE Transactions on Geoscience and Remote Sensing
  • Zhenlin Li +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.