• Home
  • Search
  • Real-Time Adaptive Partition and Resource Allocation for Multi-User End-Cloud Inference Collaboration in Mobile Environment
  • Cite Icon13
  • https://doi.org/10.1109/tmc.2024.3430103Copy DOI Icon

Real-Time Adaptive Partition and Resource Allocation for Multi-User End-Cloud Inference Collaboration in Mobile Environment

Show More
  • Abstract
  • Literature Map
  • Citations
  • Similar Papers
Abstract

The deployment of Deep Neural Networks (DNNs) requires significant computational and storage resources, which is challenging for resource-constrained end devices. To this end, collaborative deep inference is proposed, in which the DNN is divided into two parts and executed on the end device and cloud respectively. The selection of DNN partition point is the key challenge to realize end-cloud collaborative deep inference, especially in mobile environments with unstable networks. In this paper, we propose a Real-time Adaptive Partition (RAP) framework, in which a fast split point decision algorithm is proposed to realize real-time adaptive DNN model partition in the mobile network. A weighted joint optimization of DNN quantization loss, inference and transmission latency is performed. We further propose a Joint Multi-user Model Partition and Resource Allocation (JM-MPRA) algorithm under RAP framework. JM-MPRA aims to guarantee the optimized latency, accuracy and resource utilization in the multi-user scene. Experimental evaluations have demonstrated the effectiveness of RAP with JM-MPRA in improving the performance of real-time end-cloud collaborative inference in both stable and unstable mobile networks. Compared with the state-of-the-art methods, the proposed approaches can achieve up to 5.06x decrease in inference latency and bring performance improvement of 1.52% in inference accuracy.

Similar Papers
  • Conference Article
  • Citations2

Adaptive Deep Inference Framework for Cloud-Edge Collaboration

  • Aug 26, 2023
  • Ze Kou +3
  • Research Article
  • Citations21

Joint Optimization of DNN Partition and Continuous Task Scheduling for Digital Twin-Aided MEC Network With Deep Reinforcement Learning

  • Jan 01, 2023
  • IEEE Access
  • Siyu Yuan +4
  • Research Article
  • Citations77

An adaptive DNN inference acceleration framework with end–edge–cloud collaborative computing

  • Nov 04, 2022
  • Future Generation Computer Systems
  • Guozhi Liu +6
  • Research Article
  • Citations28

PArtNNer: Platform-Agnostic Adaptive Edge-Cloud DNN Partitioning for Minimizing End-to-End Latency

  • Jan 10, 2024
  • ACM Transactions on Embedded Computing Systems
  • Soumendu Kumar Ghosh +3
  • Conference Article
  • Citations208

Distributed Inference Acceleration with Adaptive DNN Partitioning and Offloading

  • Jul 01, 2020
  • Thaha Mohammed +3
  • Research Article
  • Citations26

Performance analysis of local exit for distributed deep neural networks over cloud and edge computing

  • Oct 01, 2020
  • ETRI Journal
  • Changsik Lee +3
  • Research Article
  • Citations75

Towards Real-time Cooperative Deep Inference over the Cloud and Edge End Devices

  • Jun 15, 2020
  • Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
  • Shigeng Zhang +7
  • PDF
  • Research Article
  • Citations7

LHDNN: Maintaining High Precision and Low Latency Inference of Deep Neural Networks on Encrypted Data

  • Apr 11, 2023
  • Applied Sciences
  • Jiaming Qian +5
  • Research Article

Reinforcement Learning-Based Edge-Assisted Inference with Multimodal Data

  • Jan 01, 2025
  • IEEE Transactions on Mobile Computing
  • Yu Liu +6
  • Research Article

An Ordered Aggregation-Based Ensemble Selection Method of Lightweight Deep Neural Networks With Random Initialization

  • Jan 01, 2022
  • IEEE Access
  • Lin He +2
  • Research Article

PIDC: Padding-Aware IoT Device Collaboration for Accelerating DNN Inference

  • Jan 01, 2025
  • IEEE Transactions on Mobile Computing
  • Wei Jiang +5
  • Research Article
  • Citations24

Context-aware Adaptive Surgery

  • Sep 09, 2021
  • Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
  • Hongli Wang +5
  • Research Article
  • Citations40

Optimizing Job Offloading Schedule for Collaborative DNN Inference

  • Apr 01, 2024
  • IEEE Transactions on Mobile Computing
  • Yubin Duan +1
  • Research Article
  • Citations134

DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs

  • Aug 01, 2021
  • IEEE Transactions on Computers
  • Alessio Burrello +5
  • Research Article
  • Citations6

PMP: A partition-match parallel mechanism for DNN inference acceleration in cloud–edge collaborative environments

  • Aug 18, 2023
  • Journal of Network and Computer Applications
  • Zhuofan Liao +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.