• Home
  • Search
  • PArtNNer: Platform-Agnostic Adaptive Edge-Cloud DNN Partitioning for Minimizing End-to-End Latency
  • Cite Icon28
  • https://doi.org/10.1145/3630266Copy DOI Icon

PArtNNer: Platform-Agnostic Adaptive Edge-Cloud DNN Partitioning for Minimizing End-to-End Latency

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The last decade has seen the emergence of Deep Neural Networks (DNNs) as the de facto algorithm for various computer vision applications. In intelligent edge devices, sensor data streams acquired by the device are processed by a DNN application running on either the edge device itself or in the cloud. However, “edge-only” and “cloud-only” execution of State-of-the-Art DNNs may not meet an application’s latency requirements due to the limited compute, memory, and energy resources in edge devices, dynamically varying bandwidth of edge-cloud connectivity networks, and temporal variations in the computational load of cloud servers. This work investigates distributed (partitioned) inference across edge devices (mobile/end device) and cloud servers to minimize end-to-end DNN inference latency. We study the impact of temporally varying operating conditions and the underlying compute and communication architecture on the decision of whether to run the inference solely on the edge, entirely in the cloud, or by partitioning the DNN model execution among the two. Leveraging the insights gained from this study and the wide variation in the capabilities of various edge platforms that run DNN inference, we propose PArtNNer , a platform-agnostic adaptive DNN partitioning algorithm that finds the optimal partitioning point in DNNs to minimize inference latency. PArtNNer can adapt to dynamic variations in communication bandwidth and cloud server load without requiring pre-characterization of underlying platforms. Experimental results for six image classification and object detection DNNs on a set of five commercial off-the-shelf compute platforms and three communication standards indicate that PArtNNer results in 10.2× and 3.2× (on average) and up to 21.1× and 6.7× improvements in end-to-end inference latency compared to execution of the DNN entirely on the edge device or entirely on a cloud server, respectively. Compared to pre-characterization-based partitioning approaches, PArtNNer converges to the optimal partitioning point 17.6× faster.

Similar Papers
  • Research Article
  • Citations77

An adaptive DNN inference acceleration framework with end–edge–cloud collaborative computing

  • Nov 04, 2022
  • Future Generation Computer Systems
  • Guozhi Liu +6
  • PDF
  • Conference Article
  • Citations34

FlexNN: Efficient and Adaptive DNN Inference on Memory-Constrained Edge Devices

  • May 29, 2024
  • Xiangyu Li +4
  • Research Article
  • Citations13

Real-Time Adaptive Partition and Resource Allocation for Multi-User End-Cloud Inference Collaboration in Mobile Environment

  • Dec 01, 2024
  • IEEE Transactions on Mobile Computing
  • Yiran Li +6
  • Conference Article
  • Citations208

Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

  • Jul 01, 2020
  • Thaha Mohammed +3
  • Conference Article
  • Citations2

Model Parallelism Optimization for Distributed DNN Inference on Edge Devices

  • Nov 24, 2023
  • Meng Wang +4
  • Research Article
  • Citations21

Joint Optimization of DNN Partition and Continuous Task Scheduling for Digital Twin-Aided MEC Network With Deep Reinforcement Learning

  • Jan 01, 2023
  • IEEE Access
  • Siyu Yuan +4
  • Conference Article
  • Citations5

Performance Evaluation of State-of-the-Art Edge Computing Devices for DNN Inference

  • Oct 18, 2020
  • Xalo Rancano +4
  • Conference Article
  • Citations3

An Efficient Event-driven Neuromorphic Architecture for Deep Spiking Neural Networks

  • Sep 01, 2019
  • Duy-Anh Nguyen +3
  • Conference Article
  • Citations1

Novel Adaptive DNN Partitioning Method Based on Image-Stream Pipeline Inference between the Edge and Cloud

  • May 01, 2022
  • Chenchen Ji +4
  • PDF
  • Research Article
  • Citations6

Auto-Tiler: Variable-Dimension Autoencoder with Tiling for Compressing Intermediate Feature Space of Deep Neural Networks for Internet of Things

  • Jan 29, 2021
  • Sensors (Basel, Switzerland)
  • Jeongsoo Park +2
  • Research Article
  • Citations8

Nn-METER

  • Mar 30, 2022
  • GetMobile: Mobile Computing and Communications
  • Li Lyna Zhang +5
  • Conference Article
  • Citations6

Towards Resource-aware DNN Partitioning for Edge Devices with Heterogeneous Resources

  • Dec 04, 2022
  • Muhammad Zawish +3
  • Conference Article
  • Citations5

Accelerating DNN Inference by Edge-Cloud Collaboration

  • Oct 29, 2021
  • Jianan Chen +4
  • Research Article
  • Citations113

Throughput Maximization of Delay-Aware DNN Inference in Edge Computing by Exploring DNN Model Partitioning and Inference Parallelism

  • May 01, 2023
  • IEEE Transactions on Mobile Computing
  • Jing Li +5
  • Research Article
  • Citations12

JMDC: A joint model and data compression system for deep neural networks collaborative computing in edge-cloud networks

  • Nov 21, 2022
  • Journal of Parallel and Distributed Computing
  • Yi Ding +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.