• Home
  • Search
  • RACI: A Resource-Aware Cooperative Inference Framework on Heterogeneous Edge Devices
  • https://doi.org/10.1145/3676536.3676831Copy DOI Icon

RACI: A Resource-Aware Cooperative Inference Framework on Heterogeneous Edge Devices

  • Oct 27, 2024
  • Zhenyu Wang +6 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Cooperative inference for deep neural networks (DNNs) across edge devices has received increasing attention, due to the benefits of low latency, low power consumption, and privacy preservation. Cooperative inference partitions a DNN model into multiple segments, which will then be allocated to distributed devices for parallel inference. Nonetheless, prior works fail to comprehensively study the impact of layer configurations, dynamic network bandwidths, and heterogeneous device capabilities on the inference speed, resulting in suboptimal inference performance. In this work, we conduct a comprehensive analysis of these key factors and figure out the limitations of conventional transfer-based and redundant computation-based methods. Based on the analysis, we first propose a latency prediction agent that accounts for the layer configurations, network bandwidths, and device computing capabilities, aiming to quickly evaluate the inference latency. Furthermore, we propose RACI, a resource-aware cooperative DNNs inference framework on heterogeneous edge devices. It co-trains a model agent for model partition and a workload agent for workload allocation to generate co-optimized model partition and workload allocation strategies, leading to high cooperation inference acceleration. Experimental results demonstrate that RACI outperforms the state-of-the-art approaches by 1.1× -5.2× in terms of inference speedup for three representative DNN models.

Similar Papers
  • Conference Article
  • Citations2

Model Parallelism Optimization for Distributed DNN Inference on Edge Devices

  • Nov 24, 2023
  • Meng Wang +4
  • Conference Article
  • Citations12

Parallel Detection for Efficient Video Analytics at the Edge

  • Dec 01, 2021
  • Yanzhao Wu +2
  • Conference Article
  • Citations6

Towards Resource-aware DNN Partitioning for Edge Devices with Heterogeneous Resources

  • Dec 04, 2022
  • Muhammad Zawish +3
  • Research Article
  • Citations20

Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G

  • Aug 18, 2022
  • Measurement: Sensors
  • Jyotirmoy Karjee +3
  • Research Article
  • Citations1

Enhancing Collaborative Inference on Heterogeneous Edge Devices via Adaptive Ensemble Knowledge Distillation

  • Jan 01, 2025
  • IEEE Journal on Selected Areas in Communications
  • Shangrui Wu +9
  • PDF
  • Research Article
  • Citations10

An Adaptive Task Migration Scheduling Approach for Edge‐Cloud Collaborative Inference

  • Jan 01, 2022
  • Wireless Communications and Mobile Computing
  • Boyin Zhang +4
  • Conference Article
  • Citations18

Automating Deep Neural Network Model Selection for Edge Inference

  • Dec 01, 2019
  • Bingqian Lu +3
  • Research Article
  • Citations26

FTPipeHD: A Fault-Tolerant Pipeline-Parallel Distributed Training Approach for Heterogeneous Edge Devices

  • Apr 01, 2024
  • IEEE Transactions on Mobile Computing
  • Yuhao Chen +5
  • Research Article
  • Citations30

Self-aware distributed deep learning framework for heterogeneous IoT edge devices

  • Jul 14, 2021
  • Future Generation Computer Systems
  • Yi Jin +8
  • Research Article
  • Citations28

A comparative evaluation of deep convolutional neural network and deep neural network-based land use/land cover classifications of mining regions using fused multi-sensor satellite data

  • Sep 04, 2023
  • Advances in Space Research
  • Ajay Kumar +1
  • PDF
  • Research Article
  • Citations9

A Neuron Noise-Injection Technique for Privacy Preserving Deep Neural Networks

  • Jul 01, 2020
  • Open Computer Science
  • Tosin A Adesuyi +1
  • Conference Article
  • Citations2

Latency-driven Model Placement for Efficient Edge Intelligence Service

  • Jul 01, 2022
  • Peiying Lin +4
  • Conference Article
  • Citations1

Murmuration: On-the-fly DNN Adaptation for SLO-Aware Distributed Inference in Dynamic Edge Environments

  • Aug 12, 2024
  • Jieyu Lin +3
  • Research Article
  • Citations282

CoEdge: Cooperative DNN Inference With Adaptive Workload Partitioning Over Heterogeneous Edge Devices

  • Dec 21, 2020
  • IEEE/ACM Transactions on Networking
  • Liekang Zeng +4
  • Dissertation
  • Citations1

Towards robust and secure deep learning models and beyond

  • Jan 01, 2022
  • Siyue Wang
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.