• Home
  • Search
  • Delay-Aware DNN Inference Throughput Maximization in Edge Computing via Jointly Exploring Partitioning and Parallelism
  • Cite Icon10
  • https://doi.org/10.1109/lcn52139.2021.9524928Copy DOI Icon

Delay-Aware DNN Inference Throughput Maximization in Edge Computing via Jointly Exploring Partitioning and Parallelism

  • Oct 4, 2021
  • Jing Li +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Mobile Edge Computing (MEC) has emerged as a promising paradigm catering to overwhelming explosions of mobile applications, by offloading the compute-intensive tasks to an MEC network for processing. The surging of deep learning brings new vigor and vitality to shape the prospect of intelligent Internet of Things (IoT), and edge intelligence arises to provision real-time deep neural network (DNN) inference services for users. To accelerate the processing of the DNN inference of a request in an MEC network, the DNN inference model usually can be partitioned into two connected parts: one part is processed on the local IoT device of the request; and another part is processed on a cloudlet (server) in the MEC network. Also, the DNN inference can be further accelerated by allocating multiple threads of the cloudlet in which the request is assigned.In this paper, we study a novel delay-aware DNN inference throughput maximization problem with the aim to maximize the number of delay-aware DNN service requests admitted, by accelerating each DNN inference through jointly exploring DNN model partitioning and multi-thread parallelism of DNN inference. To this end, we first show that the problem is NP-hard. We then devise a constant approximation algorithm for it. We finally evaluate the performance of the proposed algorithm through experimental simulations. Experimental results demonstrate that the proposed algorithm is promising.

Similar Papers
  • Research Article
  • Citations113

Throughput Maximization of Delay-Aware DNN Inference in Edge Computing by Exploring DNN Model Partitioning and Inference Parallelism

  • May 01, 2023
  • IEEE Transactions on Mobile Computing
  • Jing Li +5
  • Research Article
  • Citations21

Joint Optimization of DNN Partition and Continuous Task Scheduling for Digital Twin-Aided MEC Network With Deep Reinforcement Learning

  • Jan 01, 2023
  • IEEE Access
  • Siyu Yuan +4
  • Research Article
  • Citations44

IGniter: Interference-Aware GPU Resource Provisioning for Predictable DNN Inference in the Cloud

  • Mar 01, 2023
  • IEEE Transactions on Parallel and Distributed Systems
  • Fei Xu +6
  • Research Article
  • Citations146

Deep Reinforcement Learning Based Resource Management for DNN Inference in Industrial IoT

  • Mar 24, 2021
  • IEEE Transactions on Vehicular Technology
  • Weiting Zhang +6
  • Research Article
  • Citations8

PLAYS: Minimizing DNN Inference Latency in Serverless Edge Cloud for Artificial Intelligence of Things

  • Dec 01, 2024
  • IEEE Internet of Things Journal
  • Hongmin Geng +5
  • Conference Article
  • Citations281

PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning

  • Mar 09, 2020
  • Wei Niu +7
  • Book Chapter
  • Citations5

A Strategy to Accelerate the Inference of a Complex Deep Neural Network

  • Jan 01, 2023
  • P Haseena Rahmath +2
  • Research Article
  • Citations26

Performance analysis of local exit for distributed deep neural networks over cloud and edge computing

  • Oct 01, 2020
  • ETRI Journal
  • Changsik Lee +3
  • Research Article
  • Citations31

Adaptive Device-Edge Collaboration on DNN Inference in AIoT: A Digital-Twin-Assisted Approach

  • Apr 01, 2024
  • IEEE Internet of Things Journal
  • Shisheng Hu +4
  • Book Chapter
  • Citations2

MagicBatch: An Energy-Aware Scheduling Framework for DNN Inference on Heterogeneous Edge Servers in Space-Air-Ground Computation

  • Jan 01, 2023
  • Di Liu +3
  • Conference Article

Evaluating the Performance of Deep Learning Inference Service on Edge Platform

  • Oct 20, 2021
  • Hyun-Hwa Choi +5
  • Conference Article

Reinforcement Learning Based Edge-Computing DNN Inference in Power IoT Systems

  • Dec 26, 2025
  • Ying Zhou +4
  • Research Article
  • Citations69

Accelerating DNN Inference With Reliability Guarantee in Vehicular Edge Computing

  • Dec 01, 2023
  • IEEE/ACM Transactions on Networking
  • Kai Liu +4
  • Research Article
  • Citations26

Stochastic Cumulative DNN Inference With RL-Aided Adaptive IoT Device-Edge Collaboration

  • Oct 15, 2023
  • IEEE Internet of Things Journal
  • Kaige Qu +6
  • Research Article
  • Citations22

CADEC: A Combinatorial Auction for Dynamic Distributed DNN Inference Scheduling in Edge-Cloud Networks

  • Oct 01, 2025
  • IEEE Transactions on Mobile Computing
  • Xiaolong Xu +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.