• Home
  • Search
  • Efficient Memory Pool Allocation Algorithm for CNN Inference
  • https://doi.org/10.1109/hipc.2019.00049Copy DOI Icon

Efficient Memory Pool Allocation Algorithm for CNN Inference

  • Dec 1, 2019
  • Arun Abraham +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Deep Learning based applications are becoming increasingly prevalent in embedded devices. There are considerable advantages in deploying the Deep Neural Network (DNN) models directly on the device rather than on the cloud. This presents new challenges as popular networks are computationally complex and have huge memory requirements. While exciting research efforts are ongoing to improve the on-device inference performance, other concerns such as reducing the memory usage of DNNs have taken a backseat. This paper focuses specifically on how to efficiently manage the memory requirements of running Convolutional Neural Networks (CNNs) on-device. We propose a novel solution consisting of 2 steps, fine-grained buffer liveness analysis followed by memory pool allocation. We calculate the lifetime of each required buffer (including internal buffers) of a network and assign an offset to each buffer for sharing a common memory pool, resulting in efficient reuse of memory across buffers. The experimental results show that the proposed solution achieves 18x better memory usage than that of opensource Caffe framework and also up-to 22% improvement over existing state-of-the-art for an individual CNN. The scalability of the solution is showcased by a further reduction in memory usage of 31% achieved for inference pipelines that use multiple neural networks. We also show that the proposed solution achieves the theoretically minimum possible memory usage for the networks considered, using a simpler method than existing solutions.

Similar Papers
  • Research Article
  • Citations28

A comparative evaluation of deep convolutional neural network and deep neural network-based land use/land cover classifications of mining regions using fused multi-sensor satellite data

  • Sep 04, 2023
  • Advances in Space Research
  • Ajay Kumar +1
  • Research Article
  • Citations175

Deep distributed convolutional neural networks: Universality

  • Nov 01, 2018
  • Analysis and Applications
  • Ding-Xuan Zhou
  • Book Chapter
  • Citations1

Handwritten Digit Recognition Using Very Deep Convolutional Neural Network

  • Jan 01, 2022
  • M Dhilsath Fathima +2
  • Conference Article

Design and Development of Integrated Deep Convolution Neural Network Approach for Handling Heterogeneous Medical Data

  • Feb 01, 2019
  • Arifa J Shikalgar +1
  • Research Article
  • Citations1

Explainable resource-Aware IoT security model via knowledge distillation and adaptive loss function optimization

  • Mar 01, 2026
  • Expert Systems with Applications
  • Ogobuchi Daniel Okey +3
  • Conference Article
  • Citations5

Comparative analysis on Deep Convolution Neural Network models using Pytorch and OpenCV DNN frameworks for identifying optimum fruit detection solution on RISC-V architecture

  • Oct 24, 2021
  • Shalini K +3
  • Conference Article
  • Citations6

Inverse Design of Airfoils Using Convolutional Neural Network and Deep Neural Network

  • Dec 02, 2021
  • Amit Kumar +1
  • PDF
  • Research Article
  • Citations25

Research on improved convolutional wavelet neural network

  • Sep 09, 2021
  • Scientific Reports
  • Jingwei Liu +4
  • Research Article

NeMo: A Neuron-Level Modularizing-While-Training Approach for Decomposing DNN Models

  • Aug 11, 2025
  • ACM Transactions on Software Engineering and Methodology
  • Xiaohan Bi +5
  • PDF
  • Research Article
  • Citations45

Exploring Artificial Neural Networks Efficiency in Tiny Wearable Devices for Human Activity Recognition.

  • Mar 29, 2022
  • Sensors (Basel, Switzerland)
  • Emanuele Lattanzi +2
  • Research Article
  • Citations19

Feature Extraction and Classification Using Deep Convolutional Neural Networks

  • Jan 17, 2018
  • Journal of Cyber Security and Mobility
  • Jyostna Devi Bodapati +1
  • Research Article
  • Citations41

Deep neural network model for group activity recognition using contextual relationship

  • Sep 07, 2018
  • Engineering Science and Technology, an International Journal
  • S.A Vahora +1
  • Research Article
  • Citations190

Off-target predictions in CRISPR-Cas9 gene editing using deep learning.

  • Sep 01, 2018
  • Bioinformatics
  • Jiecong Lin +1
  • PDF
  • Research Article
  • Citations183

Stress detection using deep neural networks

  • Dec 01, 2020
  • BMC Medical Informatics and Decision Making
  • Russell Li +1
  • Conference Article

Absolute distance measurement based on laser self-mixing interferometry and deep neural network

  • Dec 27, 2022
  • Jinyuan Yuan +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.