• Home
  • Search
  • Automating Deep Neural Network Model Selection for Edge Inference
  • Cite Icon18
  • https://doi.org/10.1109/cogmi48466.2019.00035Copy DOI Icon

Automating Deep Neural Network Model Selection for Edge Inference

  • Dec 1, 2019
  • Bingqian Lu +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The ever increasing size of deep neural network (DNN) models once implied that they were only limited to cloud data centers for runtime inference. Nonetheless, the recent plethora of DNN model compression techniques have successfully overcome this limit, turning into a reality that DNN-based inference can be run on numerous resource-constrained edge devices including mobile phones, drones, robots, medical devices, wearables, Internet of Things devices, among many others. Naturally, edge devices are highly heterogeneous in terms of hardware specification and usage scenarios. On the other hand, compressed DNN models are so diverse that they exhibit different tradeoffs in a multi-dimension space, and not a single model can achieve optimality in terms of all important metrics such as accuracy, latency and energy consumption. Consequently, how to automatically select a compressed DNN model for an edge device to run inference with optimal quality of experience (QoE) arises as a new challenge. The state-of-the-art approaches either choose a common model for all/most devices, which is optimal for a small fraction of edge devices at best, or apply device-specific DNN model compression, which is not scalable. In this paper, by leveraging the predictive power of machine learning and keeping end users in the loop, we envision an automated device-level DNN model selection engine for QoE-optimal edge inference. To concretize our vision, we formulate the DNN model selection problem into a contextual multi-armed bandit framework, where features of edge devices and DNN models are contexts and pre-trained DNN models are arms selected online based on the history of actions and users' QoE feedback. We develop an efficient online learning algorithm to balance exploration and exploitation. Our preliminary simulation results validate our algorithm and highlight the potential of machine learning for automating DNN model selection to achieve QoE-optimal edge inference.

Similar Papers
  • Research Article
  • Citations20

Split computing: DNN inference partition with load balancing in IoT-edge platform for beyond 5G

  • Aug 18, 2022
  • Measurement: Sensors
  • Jyotirmoy Karjee +3
  • Research Article
  • Citations10

Robustness analysis and experimental validation of a deep neural network for acoustic source imaging

  • May 04, 2024
  • Mechanical Systems and Signal Processing
  • Qing Li +4
  • Research Article
  • Citations198

An Empirical Study of the Impact of Hyperparameter Tuning and Model Optimization on the Performance Properties of Deep Neural Networks

  • Apr 09, 2022
  • ACM Transactions on Software Engineering and Methodology
  • Lizhi Liao +3
  • Research Article
  • Citations13

Integrated DNN and CFD model for real-time prediction of furnace waterwall slagging of coal-fired boiler

  • Nov 28, 2024
  • Fuel
  • Hengyu Yin +5
  • Book Chapter

Active Authorization Control of Deep Models Using Channel Pruning

  • Jan 01, 2023
  • Linna Wang +3
  • Research Article
  • Citations19

A Co-Scheduling Framework for DNN Models on Mobile and Edge Devices with Heterogeneous Hardware

  • Jan 01, 2021
  • IEEE Transactions on Mobile Computing
  • Zhiyuan Xu +5
  • PDF
  • Research Article
  • Citations10

An Adaptive Task Migration Scheduling Approach for Edge‐Cloud Collaborative Inference

  • Jan 01, 2022
  • Wireless Communications and Mobile Computing
  • Boyin Zhang +4
  • Research Article
  • Citations28

A comparative evaluation of deep convolutional neural network and deep neural network-based land use/land cover classifications of mining regions using fused multi-sensor satellite data

  • Sep 04, 2023
  • Advances in Space Research
  • Ajay Kumar +1
  • Conference Article

A weight-wise watermarking technique for DNN models and its robustness against overwriting attack

  • Mar 13, 2021
  • Han He +2
  • Research Article
  • Citations15

Retrieval of wave period from altimetry: Deep learning accounting for random wave field dynamics

  • Aug 12, 2021
  • Remote Sensing of Environment
  • Jiuke Wang +2
  • Research Article
  • Citations45

Comparison of ARIMA model, DNN model and LSTM model in predicting disease burden of occupational pneumoconiosis in Tianjin, China

  • Nov 24, 2022
  • BMC Public Health
  • He-Ren Lou +3
  • Research Article
  • Citations2

A comprehensive exploration of approximate DNN models with a novel floating-point simulation framework

  • May 25, 2024
  • Performance Evaluation
  • Myeongjin Kwak +2
  • Research Article
  • Citations4

A framework for deep neural network multiuser authorization based on channel pruning

  • Apr 03, 2023
  • Concurrency and Computation: Practice and Experience
  • Linna Wang +4
  • PDF
  • Research Article
  • Citations7

Assessment of Therapeutic Responses Using a Deep Neural Network Based on 18F-FDG PET and Blood Inflammatory Markers in Pyogenic Vertebral Osteomyelitis

  • Nov 21, 2022
  • Medicina
  • Hyunkwang Shin +4
  • Book Chapter
  • Citations4

A Generalized Approach to Determine Confident Samples for Deep Neural Networks on Unseen Data

  • Jan 01, 2019
  • Min Zhang +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.