• Home
  • Search
  • Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
  • Cite Icon357
  • https://doi.org/10.1109/jproc.2022.3226481Copy DOI Icon

Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Successful integration of deep neural networks (DNNs) or deep learning (DL) has resulted in breakthroughs in many areas. However, deploying these highly accurate models for data-driven, learned, automatic, and practical machine learning (ML) solutions to end-user applications remains challenging. DL algorithms are often computationally expensive, power-hungry, and require large memory to process complex and iterative operations of millions of parameters. Hence, training and inference of DL models are typically performed on high-performance computing (HPC) clusters in the cloud. Data transmission to the cloud results in high latency, round-trip delay, security and privacy concerns, and the inability of real-time decisions. Thus, processing on edge devices can significantly reduce cloud transmission cost. Edge devices are end devices closest to the user, such as mobile phones, cyber–physical systems (CPSs), wearables, the Internet of Things (IoT), embedded and autonomous systems, and intelligent sensors. These devices have limited memory, computing resources, and power-handling capability. Therefore, optimization techniques at both the hardware and software levels have been developed to handle the DL deployment efficiently on the edge. Understanding the existing research, challenges, and opportunities is fundamental to leveraging the next generation of edge devices with artificial intelligence (AI) capability. Mainly, four research directions have been pursued for efficient DL inference on edge devices: 1) novel DL architecture and algorithm design; 2) optimization of existing DL methods; 3) development of algorithm–hardware codesign; and 4) efficient accelerator design for DL deployment. This article focuses on surveying each of the four research directions, providing a comprehensive review of the state-of-the-art tools and techniques for efficient edge inference.

Loading PDF

Similar Papers
  • Book Chapter
  • Citations15

Edge AI: Leveraging the Full Potential of Deep Learning

  • Jan 01, 2022
  • Md Maruf Hossain Shuvo
  • Conference Article
  • Citations3

Towards Ultra-Efficient DNN Inference Acceleration on Edge Devices for Wellbeing Applications

  • Jun 19, 2020
  • Yanzhi Wang
  • Dissertation

Robust and scalable deep learning for cyber-physical systems

  • Jan 01, 2021
  • Yasaman Esfandiari
  • PDF
  • Research Article
  • Citations43

Lightweight and Energy-Efficient Deep Learning Accelerator for Real-Time Object Detection on Edge Devices.

  • Jan 20, 2023
  • Sensors (Basel, Switzerland)
  • Kyungho Kim +4
  • Conference Article
  • Citations5

Characterizing Resource Heterogeneity in Edge Devices for Deep Learning Inferences

  • Jun 21, 2020
  • Jianwei Hao +3
  • Conference Article
  • Citations50

EdgeML

  • May 18, 2021
  • Zhihe Zhao +3
  • Conference Article
  • Citations13

DLASE: A light-weight framework supporting Deep Learning for Edge Devices

  • Aug 01, 2020
  • Khanh-Hoi Le Minh +2
  • Research Article
  • Citations282

DMRO: A Deep Meta Reinforcement Learning-Based Task Offloading Framework for Edge-Cloud Computing

  • Sep 01, 2021
  • IEEE Transactions on Network and Service Management
  • Guanjin Qu +3
  • PDF
  • Research Article
  • Citations9

A generic deep learning architecture optimization method for edge device based on start-up latency reduction

  • Jun 18, 2024
  • Journal of Real-Time Image Processing
  • Qi Li +2
  • Research Article
  • Citations32

Cognitive digital twin-based Internet of Robotic Things, multi-sensory extended reality and simulation modeling technologies, and generative artificial intelligence and cyber–physical manufacturing systems in the immersive industrial metaverse

  • Sep 27, 2024
  • Equilibrium. Quarterly Journal of Economics and Economic Policy
  • George Lazaroiu +15
  • PDF
  • Research Article
  • Citations7

Comprehensive Study for Breast Cancer Using Deep Learning and Traditional Machine Learning

  • Apr 12, 2022
  • ZANCO JOURNAL OF PURE AND APPLIED SCIENCES
  • Chiman Haydar Salh +1
  • Conference Article
  • Citations8

EffiE: Efficient Convolutional Neural Network for Real-Time EMG Pattern Recognition System on Edge Devices

  • Apr 24, 2023
  • Jimmy Lu +4
  • Conference Article
  • Citations2

Model Parallelism Optimization for Distributed DNN Inference on Edge Devices

  • Nov 24, 2023
  • Meng Wang +4
  • Research Article
  • Citations2

Development of a deep learning algorithm for detecting significant coronary artery stenosis in whole-heart coronary magnetic resonance angiography

  • Jan 01, 2025
  • Journal of Cardiovascular Magnetic Resonance
  • Masafumi Takafuji +11
  • PDF
  • Research Article
  • Citations107

Intrusion Detection System for Industrial Internet of Things Based on Deep Reinforcement Learning

  • Jan 01, 2022
  • Wireless Communications and Mobile Computing
  • Sumegh Tharewal +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.