• Home
  • Search
  • Compressing medical deep neural network models for edge devices using knowledge distillation
  • Cite Icon22
  • https://doi.org/10.1016/j.jksuci.2023.101616Copy DOI Icon

Compressing medical deep neural network models for edge devices using knowledge distillation

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Recently, deep neural networks (DNNs) have been used successfully in many fields, particularly, in medical diagnosis. However, deep learning (DL)modelsare expensive in terms of memory and computing resources, whichhinders their implementation in limited-resources devices or for delay-sensitive systems. Therefore, these deep models need to be accelerated and compressed to smaller sizes to be deployed on edge devices without noticeably affecting their performance. In this paper, recent accelerating and compression approaches of DNN are analyzed and compared regarding their performance, applications, benefits, and limitations with a more focus on the knowledge distillation approach as a successful emergent approach in this field. In addition, a framework is proposed to develop knowledge distilled DNN models that can be deployed on fog/edge devices for automatic disease diagnosis. To evaluate the proposed framework, two compressed medical diagnosis systems are proposed based on knowledge distillation deep neural models for both COVID-19 and Malaria. The experimental results show that these knowledge distilled models have been compressed by 18.4% and 15% of the original model and their responses accelerated by 6.14x and 5.86%, respectively, while there were no significant drop in their performance (dropped by 0.9% and 1.2%, respectively). Furthermore, the distilled models are compared with other pruned and quantized models. The obtained results revealed the superiority of the distilled models in terms of compression rates and response time.

Similar Papers
  • Research Article
  • Citations1

A Novel Click-Through Rate Prediction Model Based on Deep Feature Fusion Network

  • Jan 31, 2023
  • AATCC Journal of Research
  • Xiujin Shi +3
  • PDF
  • Research Article
  • Citations6

A deep learning-based method for predicting the low-cycle fatigue life of austenitic stainless steel

  • Aug 01, 2023
  • Materials Research Express
  • Hongyan Duan +5
  • Research Article
  • Citations3

Magnetoencephalography Decoding Transfer Approach: From Deep Learning Models to Intrinsically Interpretable Models.

  • May 01, 2024
  • IEEE journal of biomedical and health informatics
  • Yongdong Fan +3
  • Research Article
  • Citations885

Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks.

  • Jan 29, 2021
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Lin Wang +1
  • PDF
  • Research Article
  • Citations6

Multi-branch deep learning neural network prediction model for the development of angular biosensors based on sEMG

  • Oct 11, 2024
  • Frontiers in Bioengineering and Biotechnology
  • Liman Yang +6
  • Book Chapter
  • Citations6

On the Performance of Classic and Deep Neural Models in Image Recognition

  • Jan 01, 2017
  • Ricardo García-Ródenas +2
  • PDF
  • Research Article
  • Citations1

Predicting an opaque bubble layer during small-incision lenticule extraction surgery based on deep learning

  • Oct 30, 2024
  • Frontiers in Cell and Developmental Biology
  • Zeyu Zhu +9
  • Research Article
  • Citations126

DeepVID: Deep Visual Interpretation and Diagnosis for Image Classifiers via Knowledge Distillation.

  • Mar 15, 2019
  • IEEE Transactions on Visualization and Computer Graphics
  • Junpeng Wang +4
  • PDF
  • Research Article
  • Citations2

DDLS: Distributed Deep Learning Systems: A Review

  • Apr 28, 2021
  • Türk bilgisayar ve matematik eğitimi dergisi
  • Najdavan Abduljawad Kako, Et Al
  • Research Article
  • Citations34

Development and Validation of a Deep Neural Network for Accurate Identification of Endoscopic Images From Patients With Ulcerative Colitis and Crohn's Disease

  • Mar 18, 2022
  • Frontiers in Medicine
  • Guangcong Ruan +28
  • Research Article
  • Citations12

Evaluating The Performance Of Deep Neural Networks For Health Decision Making

  • Jan 01, 2018
  • Procedia Computer Science
  • Emelia Opoku Aboagye +2
  • Conference Article
  • Citations1

Intelligent Cybersecurity Situational Awareness Model Based on Deep Neural Network

  • Oct 01, 2020
  • Chuang Ma +3
  • PDF
  • Research Article
  • Citations13

A Theoretical Framework Towards Building a Lightweight Model for Pothole Detection using Knowledge Distillation Approach

  • Jan 01, 2022
  • SHS Web of Conferences
  • Aminu Musa +2
  • Research Article
  • Citations12

A 10-Year Probability Deep Neural Network Prediction Model for Lung Cancer

  • Feb 23, 2021
  • Cancers
  • Hsiu-An Lee +2
  • Research Article
  • Citations22

Rethinking the image feature biases exhibited by deep convolutional neural network models in image recognition

  • May 04, 2022
  • CAAI Transactions on Intelligence Technology
  • Dawei Dai +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.