• Home
  • Search
  • Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading
  • Open Access IconOpen Access
  • Cite Icon208
  • https://doi.org/10.1109/infocom41043.2020.9155237Copy DOI Icon

Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

  • Jul 1, 2020
  • Thaha Mohammed +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Deep neural networks (DNN) are the de-facto solution behind many intelligent applications of today, ranging from machine translation to autonomous driving. DNNs are accurate but resource-intensive, especially for embedded devices such as mobile phones and smart objects in the Internet of Things. To overcome the related resource constraints, DNN inference is generally offloaded to the edge or to the cloud. This is accomplished by partitioning the DNN and distributing computations at the two different ends. However, most of existing solutions simply split the DNN into two parts, one running locally or at the edge, and the other one in the cloud. In contrast, this article proposes a technique to divide a DNN in multiple partitions that can be processed locally by end devices or offloaded to one or multiple powerful nodes, such as in fog networks. The proposed scheme includes both an adaptive DNN partitioning scheme and a distributed algorithm to offload computations based on a matching game approach. Results obtained by using a self-driving car dataset and several DNN benchmarks show that the proposed solution significantly reduces the total latency for DNN inference compared to other distributed approaches and is 2.6 to 4.2 times faster than the state of the art.

Similar Papers
  • Research Article
  • Citations21

Joint Optimization of DNN Partition and Continuous Task Scheduling for Digital Twin-Aided MEC Network With Deep Reinforcement Learning

  • Jan 01, 2023
  • IEEE Access
  • Siyu Yuan +4
  • Research Article
  • Citations77

An adaptive DNN inference acceleration framework with end–edge–cloud collaborative computing

  • Nov 04, 2022
  • Future Generation Computer Systems
  • Guozhi Liu +6
  • Research Article
  • Citations13

Real-Time Adaptive Partition and Resource Allocation for Multi-User End-Cloud Inference Collaboration in Mobile Environment

  • Dec 01, 2024
  • IEEE Transactions on Mobile Computing
  • Yiran Li +6
  • Research Article
  • Citations28

PArtNNer: Platform-Agnostic Adaptive Edge-Cloud DNN Partitioning for Minimizing End-to-End Latency

  • Jan 10, 2024
  • ACM Transactions on Embedded Computing Systems
  • Soumendu Kumar Ghosh +3
  • Dissertation

Latency-Critical Inference Serving for Deep Learning

  • Sep 20, 2023
  • Vinod Vijay Nigade
  • Book Chapter
  • Citations6

DNN Inference Acceleration with Partitioning and Early Exiting in Edge Computing

  • Jan 01, 2021
  • Chao Li +4
  • Research Article
  • Citations113

Throughput Maximization of Delay-Aware DNN Inference in Edge Computing by Exploring DNN Model Partitioning and Inference Parallelism

  • May 01, 2023
  • IEEE Transactions on Mobile Computing
  • Jing Li +5
  • Conference Article
  • Citations10

Delay-Aware DNN Inference Throughput Maximization in Edge Computing via Jointly Exploring Partitioning and Parallelism

  • Oct 04, 2021
  • Jing Li +4
  • Research Article

PIDC: Padding-Aware IoT Device Collaboration for Accelerating DNN Inference

  • Jan 01, 2025
  • IEEE Transactions on Mobile Computing
  • Wei Jiang +5
  • Research Article
  • Citations146

Deep Reinforcement Learning Based Resource Management for DNN Inference in Industrial IoT

  • Mar 24, 2021
  • IEEE Transactions on Vehicular Technology
  • Weiting Zhang +6
  • Research Article
  • Citations26

Performance analysis of local exit for distributed deep neural networks over cloud and edge computing

  • Oct 01, 2020
  • ETRI Journal
  • Changsik Lee +3
  • Conference Article
  • Citations6

Design an Efficient DNN Inference Framework with PS-PL Synergies in FPGA for Edge Computing

  • Nov 25, 2022
  • Wancheng Liu +4
  • Conference Article
  • Citations281

PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning

  • Mar 09, 2020
  • Wei Niu +7
  • Conference Article
  • Citations5

Performance Evaluation of State-of-the-Art Edge Computing Devices for DNN Inference

  • Oct 18, 2020
  • Xalo Rancano +4
  • PDF
  • Research Article
  • Citations6

Auto-Tiler: Variable-Dimension Autoencoder with Tiling for Compressing Intermediate Feature Space of Deep Neural Networks for Internet of Things

  • Jan 29, 2021
  • Sensors (Basel, Switzerland)
  • Jeongsoo Park +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.