• Home
  • Search
  • HNPU: An Adaptive DNN Training Processor Utilizing Stochastic Dynamic Fixed-Point and Active Bit-Precision Searching
  • Cite Icon59
  • https://doi.org/10.1109/jssc.2021.3066400Copy DOI Icon

HNPU: An Adaptive DNN Training Processor Utilizing Stochastic Dynamic Fixed-Point and Active Bit-Precision Searching

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

This article presents HNPU, which is an energy-efficient deep neural network (DNN) training processor by adopting algorithm-hardware co-design. The HNPU supports stochastic dynamic fixed-point representation and layer-wise adaptive precision searching unit for low-bit-precision training. It additionally utilizes slice-level reconfigurability and sparsity to maximize its efficiency both in DNN inference and training. Adaptive bandwidth reconfigurable accumulation network enables reconfigurable DNN allocation and maintains its high core utilization even in various bit-precision conditions. Fabricated in a 28-nm process, the HNPU accomplished at least <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5.9\times $ </tex-math></inline-formula> higher energy efficiency and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.5\times $ </tex-math></inline-formula> higher area efficiency in actual DNN training compared with the previous state-of-the-art on-chip learning processors.

Similar Papers
  • Research Article
  • Citations179

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

  • May 04, 2021
  • IEEE Transactions on Artificial Intelligence
  • Edgar Galván +1
  • Conference Article
  • Citations15

PCM: Precision-Controlled Memory System for Energy Efficient Deep Neural Network Training

  • Mar 01, 2020
  • Boyeal Kim +9
  • Conference Article
  • Citations8

A Framework for Distributed Deep Neural Network Training with Heterogeneous Computing Platforms

  • Dec 01, 2019
  • Bontak Gu +3
  • Research Article
  • Citations52

TxSim: Modeling Training of Deep Neural Networks on Resistive Crossbar Systems

  • Mar 31, 2021
  • IEEE Transactions on Very Large Scale Integration (VLSI) Systems
  • Sourjya Roy +3
  • Conference Article
  • Citations3

ADA-GP: Accelerating DNN Training By Adaptive Gradient Prediction

  • Oct 28, 2023
  • Vahid Janfaza +3
  • Research Article
  • Citations6

A Guessing Entropy-Based Framework for Deep Learning-Assisted Side-Channel Analysis

  • Jan 01, 2023
  • IEEE Transactions on Information Forensics and Security
  • Ziyue Zhang +2
  • Conference Article
  • Citations2

Cannikin: Optimal Adaptive Distributed DNN Training over Heterogeneous Clusters

  • Dec 02, 2024
  • Chengyi Nie +2
  • Conference Article
  • Citations4

MalleTrain: Deep Neural Networks Training on Unfillable Supercomputer Nodes

  • May 07, 2024
  • Xiaolong Ma +6
  • Dissertation

A novel model quantization scheme for efficient deep neural networks inference and training

  • Jan 01, 2024
  • Sung-En Chang
  • PDF
  • Research Article
  • Citations46

An Overview of Energy-Efficient Hardware Accelerators for On-Device Deep-Neural-Network Training

  • Jan 01, 2021
  • IEEE Open Journal of the Solid-State Circuits Society
  • Jinsu Lee +1
  • Research Article
  • Citations24

ApproxTrain: Fast Simulation of Approximate Multipliers for DNN Training and Inference

  • Nov 01, 2023
  • IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
  • Jing Gong +5
  • Conference Article
  • Citations3

AccDP: Accelerated Data-Parallel Distributed DNN Training for Modern GPU-Based HPC Clusters

  • Dec 01, 2022
  • Nawras Alnaasan +4
  • Research Article
  • Citations7

A convergence analysis of Nesterov’s accelerated gradient method in training deep linear neural networks

  • Sep 05, 2022
  • Information Sciences
  • Xin Liu +2
  • Research Article
  • Citations1

Constructed encoded data based coded distributed DNN training for edge computing scenario

  • Sep 24, 2024
  • Physical Communication
  • Mingzhu Hu +2
  • Research Article

Efficient Deep Neural Network Training With a Novel 5.3-Bit Block Floating Point Data Format

  • Jun 01, 2025
  • IEEE Transactions on Circuits and Systems for Artificial Intelligence
  • Mohammad Hassani Sadi +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.