• Home
  • Search
  • Model Parallelism Optimization for Distributed DNN Inference on Edge Devices
  • Cite Icon2
  • https://doi.org/10.1109/paap60200.2023.10391646Copy DOI Icon

Model Parallelism Optimization for Distributed DNN Inference on Edge Devices

  • Nov 24, 2023
  • Meng Wang +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Deep neural networks (DNNs) have recently gained widespread application in various domains. However, the computational and memory requirements of DNN models pose challenges for deploying them on resource-constrained edge devices. With the widespread use of the Internet of Things (IoT), heterogeneous edge devices with diverse computational capabilities and network conditions are increasingly employed for DNN inference. This paper proposes a distributed DNN model deployment scheme for edge device clusters. The DNN model partitioning is performed using two algorithms: Edge Layer Partitioning (EdgeLP), which partitions a single neural network layer, and Edge Model Partitioning (EdgeMP), which performs complete model partitioning. These algorithms consider both the computational capabilities and network conditions of the edge devices. To address the transmission overhead between collaborative edge devices, layer fusion and data quantization are applied to reduce the amount of transmitted data. Experimental results show that our method dramatically improves the performance of distributed DNN inference in heterogeneous scenarios. Specifically, on a cluster of three edge devices, the proposed scheme achieves DNN inference time acceleration speedup of 1.38–1.72× without accuracy loss compared to a state-of-the-art scheme.

Similar Papers
  • Research Article
  • Citations20

Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G

  • Aug 18, 2022
  • Measurement: Sensors
  • Jyotirmoy Karjee +3
  • Conference Article
  • Citations1

Murmuration: On-the-fly DNN Adaptation for SLO-Aware Distributed Inference in Dynamic Edge Environments

  • Aug 12, 2024
  • Jieyu Lin +3
  • Research Article
  • Citations282

CoEdge: Cooperative DNN Inference With Adaptive Workload Partitioning Over Heterogeneous Edge Devices

  • Dec 21, 2020
  • IEEE/ACM Transactions on Networking
  • Liekang Zeng +4
  • Conference Article
  • Citations6

Towards Resource-aware DNN Partitioning for Edge Devices with Heterogeneous Resources

  • Dec 04, 2022
  • Muhammad Zawish +3
  • Conference Article
  • Citations15

EdgeLD: Locally Distributed Deep Learning Inference on Edge Device Clusters

  • Dec 01, 2020
  • Feng Xue +5
  • Conference Article

RACI: A Resource-Aware Cooperative Inference Framework on Heterogeneous Edge Devices

  • Oct 27, 2024
  • Zhenyu Wang +6
  • Conference Article
  • Citations1

Split Computing based Quantized PoseNet Model for Distributed AI Architecture in IoT-Edge

  • Jan 03, 2023
  • Jyotirmoy Karjee +5
  • Conference Article
  • Citations50

EdgeML

  • May 18, 2021
  • Zhihe Zhao +3
  • Conference Article
  • Citations18

Automating Deep Neural Network Model Selection for Edge Inference

  • Dec 01, 2019
  • Bingqian Lu +3
  • Research Article
  • Citations113

Throughput Maximization of Delay-Aware DNN Inference in Edge Computing by Exploring DNN Model Partitioning and Inference Parallelism

  • May 01, 2023
  • IEEE Transactions on Mobile Computing
  • Jing Li +5
  • Conference Article
  • Citations10

Delay-Aware DNN Inference Throughput Maximization in Edge Computing via Jointly Exploring Partitioning and Parallelism

  • Oct 04, 2021
  • Jing Li +4
  • Research Article
  • Citations146

Deep Reinforcement Learning Based Resource Management for DNN Inference in Industrial IoT

  • Mar 24, 2021
  • IEEE Transactions on Vehicular Technology
  • Weiting Zhang +6
  • PDF
  • Research Article
  • Citations10

An Adaptive Task Migration Scheduling Approach for Edge‐Cloud Collaborative Inference

  • Jan 01, 2022
  • Wireless Communications and Mobile Computing
  • Boyin Zhang +4
  • Conference Article
  • Citations2

Latency-driven Model Placement for Efficient Edge Intelligence Service

  • Jul 01, 2022
  • Peiying Lin +4
  • Research Article
  • Citations6

PMP: A partition-match parallel mechanism for DNN inference acceleration in cloud–edge collaborative environments

  • Aug 18, 2023
  • Journal of Network and Computer Applications
  • Zhuofan Liao +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.