• Home
  • Search
  • PCM: Precision-Controlled Memory System for Energy Efficient Deep Neural Network Training
  • Cite Icon15
  • https://doi.org/10.23919/date48585.2020.9116530Copy DOI Icon

PCM: Precision-Controlled Memory System for Energy Efficient Deep Neural Network Training

  • Mar 1, 2020
  • Boyeal Kim +9 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Deep neural network (DNN) training suffers from the significant energy consumption in memory system, and most existing energy reduction techniques for memory system have focused on introducing low precision that is compatible with computing unit (e.g., FP16, FP8). These researches have shown that even in learning the networks with FP16 data precision, it is possible to provide training accuracy as good as FP32, de facto standard of the DNN training. However, our extensive experiments show that we can further reduce the data precision while maintaining the training accuracy of DNNs, which can be obtained by truncating some least significant bits (LSBs) of FP16, named as hard approximation. Nevertheless, the existing hard-ware structures for DNN training cannot efficiently support such low precision. In this work, we propose a novel memory system architecture for GPUs, named as precision-controlled memory system (PCM), which allows for flexible management at the level of hard approximation. PCM provides high DRAM bandwidth by distributing each precision to different channels with as transposed data mapping on DRAM. In addition, PCM supports fine-grained hard approximation in the L1 data cache using software-controlled registers, which can reduce data movement and thereby improve energy saving and system performance. Furthermore, PCM facilitates the reduction of data maintenance energy, which accounts for a considerable portion of memory energy consumption, by controlling refresh period of DRAM. The experimental results show that in training CIFAR-100 dataset on Resnet-20 with precision tuning, PCM achieves energy saving and performance enhancement by 66% and 20%, respectively, without loss of accuracy.

Similar Papers
  • Research Article
  • Citations59

HNPU: An Adaptive DNN Training Processor Utilizing Stochastic Dynamic Fixed-Point and Active Bit-Precision Searching

  • Mar 24, 2021
  • IEEE Journal of Solid-State Circuits
  • Donghyeon Han +6
  • Research Article
  • Citations179

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

  • May 04, 2021
  • IEEE Transactions on Artificial Intelligence
  • Edgar Galván +1
  • Conference Article
  • Citations3

ADA-GP: Accelerating DNN Training By Adaptive Gradient Prediction

  • Oct 28, 2023
  • Vahid Janfaza +3
  • Conference Article
  • Citations8

A Framework for Distributed Deep Neural Network Training with Heterogeneous Computing Platforms

  • Dec 01, 2019
  • Bontak Gu +3
  • Research Article
  • Citations6

A Guessing Entropy-Based Framework for Deep Learning-Assisted Side-Channel Analysis

  • Jan 01, 2023
  • IEEE Transactions on Information Forensics and Security
  • Ziyue Zhang +2
  • Research Article
  • Citations30

DLUX: A LUT-Based Near-Bank Accelerator for Data Center Deep Learning Training Workloads

  • Aug 01, 2021
  • IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
  • Peng Gu +6
  • Conference Article
  • Citations4

MalleTrain: Deep Neural Networks Training on Unfillable Supercomputer Nodes

  • May 07, 2024
  • Xiaolong Ma +6
  • Conference Article
  • Citations3

AccDP: Accelerated Data-Parallel Distributed DNN Training for Modern GPU-Based HPC Clusters

  • Dec 01, 2022
  • Nawras Alnaasan +4
  • PDF
  • Research Article
  • Citations46

An Overview of Energy-Efficient Hardware Accelerators for On-Device Deep-Neural-Network Training

  • Jan 01, 2021
  • IEEE Open Journal of the Solid-State Circuits Society
  • Jinsu Lee +1
  • Research Article
  • Citations52

TxSim: Modeling Training of Deep Neural Networks on Resistive Crossbar Systems

  • Mar 31, 2021
  • IEEE Transactions on Very Large Scale Integration (VLSI) Systems
  • Sourjya Roy +3
  • Conference Article
  • Citations27

MINT: Mixed-Precision RRAM-Based IN-Memory Training Architecture

  • Oct 01, 2020
  • Hongwu Jiang +3
  • Conference Article
  • Citations2

Cannikin: Optimal Adaptive Distributed DNN Training over Heterogeneous Clusters

  • Dec 02, 2024
  • Chengyi Nie +2
  • Research Article
  • Citations2

Accelerating deep network training for radar identification using batch normalization

  • Nov 25, 2024
  • Defence Science Journal
  • Preeti Gupta +2
  • Research Article
  • Citations7

A convergence analysis of Nesterov’s accelerated gradient method in training deep linear neural networks

  • Sep 05, 2022
  • Information Sciences
  • Xin Liu +2
  • Conference Article
  • Citations7

Comparison of Loss Functions for Training of Deep Neural Networks in Shogi

  • Nov 01, 2018
  • Hanhua Zhu +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.