• Home
  • Search
  • Distributed Inference with Deep Learning Models across Heterogeneous Edge Devices
  • Cite Icon72
  • https://doi.org/10.1109/infocom48880.2022.9796896Copy DOI Icon

Distributed Inference with Deep Learning Models across Heterogeneous Edge Devices

  • May 2, 2022
  • Chenghao Hu +1 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Recent years witnessed an increasing research attention in deploying deep learning models on edge devices for inference. Due to limited capabilities and power constraints, it may be necessary to distribute the inference workload across multiple devices. Existing mechanisms divided the model across edge devices with the assumption that deep learning models are constructed with a chain of layers. In reality, however, modern deep learning models are more complex, involving a directed acyclic graph (DAG) rather than a chain of layers.In this paper, we present EdgeFlow, a new distributed inference mechanism designed for general DAG structured deep learning models. Specifically, EdgeFlow partitions model layers into independent execution units with a new progressive model partitioning algorithm. By producing near-optimal model partitions, our new algorithm seeks to improve the run-time performance of distributed inference as these partitions are distributed across the edge devices. During inference, EdgeFlow orchestrates the intermediate results flowing through these units to fulfill the complicated layer dependencies. We have implemented Edge-Flow based on PyTorch, and evaluated it with state-of-the-art deep learning models in different structures. The results show that EdgeFlow reducing the inference latency by up to 40.2% compared with other approaches, which demonstrates the effectiveness of our design.

Similar Papers
  • Conference Article
  • Citations5

FedPruNet: Federated Learning Using Pruning Neural Network

  • Jul 01, 2022
  • L Gowtham +2
  • Conference Article
  • Citations7

Deep-Edge: An Efficient Framework for Deep Learning Model Update on Heterogeneous Edge

  • May 01, 2020
  • Anirban Bhattacharjee +6
  • Conference Article
  • Citations2

Model Parallelism Optimization for Distributed DNN Inference on Edge Devices

  • Nov 24, 2023
  • Meng Wang +4
  • Conference Article

RACI: A Resource-Aware Cooperative Inference Framework on Heterogeneous Edge Devices

  • Oct 27, 2024
  • Zhenyu Wang +6
  • Research Article
  • Citations30

Self-aware distributed deep learning framework for heterogeneous IoT edge devices

  • Jul 14, 2021
  • Future Generation Computer Systems
  • Yi Jin +8
  • Conference Article
  • Citations1

Towards resource-efficient detection-driven processing of multi-stream videos

  • Oct 25, 2021
  • Md Adnan Arefeen +1
  • Conference Article
  • Citations435

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data

  • Dec 10, 2020
  • Yujing Chen +3
  • Research Article

Efficient FPGA accelerator for low-power high-speed BCI motor imagery classification using novel deep learning.

  • Jun 02, 2026
  • Neural networks : the official journal of the International Neural Network Society
  • Saravanakumar C +2
  • Research Article

Characterizing Hardware Utilization on Edge Devices when Inferring Compressed Deep Learning Models

  • Nov 06, 2024
  • MATRIK : Jurnal Manajemen, Teknik Informatika dan Rekayasa Komputer
  • Ahmad Naufal Labiib Nabhaan +2
  • Conference Article
  • Citations1

MobileDLSearch: Ontology-based Mobile Platform for Effective Sharing and Reuse of Deep Learning Models

  • Dec 01, 2021
  • Zhangcheng Qiang +5
  • Research Article
  • Citations15

Memory optimization at Edge for Distributed Convolution Neural Network

  • Sep 15, 2022
  • Transactions on Emerging Telecommunications Technologies
  • Soumyalatha Naveen +1
  • Research Article
  • Citations3

Federated Recognition Architecture Based on Voting and Feedback Mechanisms for Accuracy Object Classification in Distributed Edge Intelligence Environment

  • Nov 15, 2023
  • IEEE Sensors Journal
  • Nguyen Anh Tuan +3
  • Conference Article
  • Citations7

Exploring Decentralized Collaboration in Heterogeneous Edge Training

  • Nov 01, 2020
  • Xiang Chen +1
  • Research Article
  • Citations1

Enhancing Collaborative Inference on Heterogeneous Edge Devices via Adaptive Ensemble Knowledge Distillation

  • Jan 01, 2025
  • IEEE Journal on Selected Areas in Communications
  • Shangrui Wu +9
  • Research Article
  • Citations31

Efficient Hardware Acceleration Techniques for Deep Learning on Edge Devices: A Comprehensive Performance Analysis

  • Aug 01, 2023
  • KHWARIZMIA
  • M.A Burhanuddin
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.