• Home
  • Search
  • ESCEPE: Early-Exit Network Section-Wise Model Compression Using Self-distillation and Weight Clustering
  • Cite Icon5
  • https://doi.org/10.1145/3578354.3592872Copy DOI Icon

ESCEPE: Early-Exit Network Section-Wise Model Compression Using Self-distillation and Weight Clustering

  • May 8, 2023
  • Saeed Khalilian Gourtani +1 more
Show More
  • Abstract
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Deploying deep learning models on resource-constrained (edge) devices is challenging due to their high computational demands and large model sizes. Early-exit neural networks are one of the approaches to make deep learning models more efficient for resource-constrained devices by reducing computational cost and latency. However, even with early-exit neural networks, the model size may remain a problem when deploying them on edge devices. To address this problem, we propose a section-wise model compression technique for compressing an early-exit neural network with intermediate classifiers. Our approach divides the model into a few sections and uses different compression settings in the weight clustering-based compression for each section to prevent accuracy loss in the intermediate sections. We demonstrate that knowledge distillation can be used in the retraining phase to transfer knowledge from uncompressed to compressed sections and to accelerate the recovery of performance reduction after the weight clustering stages. The performance evaluation of our proposed method on CIFAR10 and CIFAR100 datasets using ResNet and WideResNet architectures demonstrates that the proposed technique can compress an early-exit neural network with a high compression ratio with minimal impact on the accuracy of intermediate classifiers. The proposed method achieves compression ratios of more than 36 and 22 times for ResNet18 with three shallow classifiers on CIFAR10 and CIFAR100, respectively, with an ensemble accuracy loss of less than 1%. By eliminating shallow classifiers from the early-exit model, the static model can achieve compression ratios of up to 64 and 52 times for ResNet18 and WideResNet50, respectively, on the CIFAR10 dataset with an accuracy loss of less than 2.5%.

Loading PDF

Similar Papers
  • Research Article
  • Citations41

EdgeKE: An On-Demand Deep Learning IoT System for Cognitive Big Data on Industrial Edge Devices

  • Dec 15, 2020
  • IEEE Transactions on Industrial Informatics
  • Weiwei Fang +4
  • Research Article
  • Citations15

Federated learning by employing knowledge distillation on edge devices with limited hardware resources

  • Feb 14, 2023
  • Neurocomputing
  • Ehsan Tanghatari +3
  • Conference Article
  • Citations5

FedPruNet: Federated Learning Using Pruning Neural Network

  • Jul 01, 2022
  • L Gowtham +2
  • PDF
  • Research Article
  • Citations5

Effective Online Knowledge Distillation via Attention-Based Model Ensembling

  • Nov 16, 2022
  • Mathematics
  • Diana-Laura Borza +3
  • Research Article
  • Citations2

PCA-Pruner: Filter pruning by principal component analysis

  • Aug 10, 2022
  • Journal of Intelligent & Fuzzy Systems
  • Wei Zhang +1
  • Conference Article
  • Citations43

Preventing Catastrophic Forgetting and Distribution Mismatch in Knowledge Distillation via Synthetic Data

  • Jan 01, 2022
  • Kuluhan Binici +3
  • Conference Article

Edge AI Acceleration on Ultra96 SoC: Rapid Model-to-FPGA Hardware Mapping Using NNgen

  • Dec 17, 2025
  • Swati +2
  • Research Article
  • Citations2

Web-aided data set expansion in deep learning: evaluating trainable activation functions in ResNet for improved image classification

  • Jul 12, 2024
  • International Journal of Web Information Systems
  • Zhiqiang Zhang +5
  • PDF
  • Research Article
  • Citations13

A Theoretical Framework Towards Building a Lightweight Model for Pothole Detection using Knowledge Distillation Approach

  • Jan 01, 2022
  • SHS Web of Conferences
  • Aminu Musa +2
  • Book Chapter
  • Citations10

Enabling Inference and Training of Deep Learning Models for AI Applications on IoT Edge Devices

  • Jan 01, 2022
  • Divyasheel Sharma +1
  • Research Article
  • Citations2

Learning sparse convolutional neural networks through filter pruning for efficient fault diagnosis on edge devices

  • Mar 02, 2025
  • Nondestructive Testing and Evaluation
  • Gaowei Xu +2
  • Research Article
  • Citations3

Domain-Adaptive Online Active Learning for Real-Time Intelligent Video Analytics on Edge Devices

  • Nov 01, 2024
  • IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
  • Michele Boldo +4
  • Research Article
  • Citations17

Lightweight Deep Learning Models For Edge Devices—A Survey

  • Jan 06, 2025
  • International Journal of Computer Information Systems and Industrial Management Applications
  • Aminu Musa +5
  • Research Article
  • Citations8

Lightweight Deep Learning Models For Edge Devices—A Survey

  • Jan 06, 2025
  • International Journal of Computer Information Systems and Industrial Management Applications
  • Aminu Musa +5
  • Conference Article
  • Citations3

FedKDShap: Enhancing Federated Learning via Shapley Values Driven Knowledge Distillation on Non-IID Data

  • May 08, 2025
  • Nazmus Shakib Shadin +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.