• Home
  • Search
  • Lightening the Load with Highly Accurate Storage- and Energy-Efficient LightNNs
  • Cite Icon15
  • https://doi.org/10.1145/3270689Copy DOI Icon

Lightening the Load with Highly Accurate Storage- and Energy-Efficient LightNNs

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Hardware implementations of deep neural networks (DNNs) have been adopted in many systems because of their higher classification speed. However, while they may be characterized by better accuracy, larger DNNs require significant energy and area, thereby limiting their wide adoption. The energy consumption of DNNs is driven by both memory accesses and computation. Binarized neural networks (BNNs), as a tradeoff between accuracy and energy consumption, can achieve great energy reduction and have good accuracy for large DNNs due to their regularization effect. However, BNNs show poor accuracy when a smaller DNN configuration is adopted. In this article, we propose a new DNN architecture, LightNN, which replaces the multiplications to one shift or a constrained number of shifts and adds. Our theoretical analysis for LightNNs shows that their accuracy is maintained while dramatically reducing storage and energy requirements. For a fixed DNN configuration, LightNNs have better accuracy at a slight energy increase than BNNs, yet are more energy efficient with only slightly less accuracy than conventional DNNs. Therefore, LightNNs provide more options for hardware designers to trade off accuracy and energy. Moreover, for large DNN configurations, LightNNs have a regularization effect, making them better in accuracy than conventional DNNs. These conclusions are verified by experiment using the MNIST and CIFAR-10 datasets for different DNN configurations. Our FPGA implementation for conventional DNNs and LightNNs confirms all theoretical and simulation results and shows that LightNNs reduce latency and use fewer FPGA resources compared to conventional DNN architectures.

Similar Papers
  • Conference Article
  • Citations40

LightNN

  • May 10, 2017
  • Ruizhou Ding +4
  • Supplementary Content
  • Citations1

Improving Efficiency and Accuracy for Training and Inference of Hardware-aware Machine Learning Systems

  • Mar 12, 2020
  • Figshare
  • Ruixue Ding
  • Conference Article
  • Citations26

Quantized deep neural networks for energy efficient hardware-based inference

  • Jan 22, 2018
  • Ruixue Ding +3
  • Conference Article
  • Citations49

Quantized deep neural networks for energy efficient hardware-based inference

  • Jan 01, 2018
  • Ruizhou Ding +3
  • Book Chapter

Evolutionary-Based Co-optimization of DNN and Hardware Configurations on Edge GPU

  • Jan 01, 2022
  • Halima Bouzidi +4
  • Conference Article
  • Citations3

Efficient Deep Feature Learning and Extraction via StochasticNets

  • Jun 01, 2016
  • Mohammad Javad Shafiee +3
  • Research Article
  • Citations38

StochasticNet: Forming Deep Neural Networks via Stochastic Connectivity

  • Jan 01, 2016
  • IEEE Access
  • Mohammad Javad Shafiee +2
  • Research Article
  • Citations22

Dual-configuration in-memory computing bitcells using SiOx RRAM for binary neural networks

  • Jan 17, 2022
  • Applied Physics Letters
  • Sandeep Kaur Kingra +8
  • Research Article
  • Citations17

An Empirical Approach to Enhance Performance for Scalable CORDIC-Based Deep Neural Networks

  • Jun 20, 2023
  • ACM Transactions on Reconfigurable Technology and Systems
  • Gopal Raut +2
  • PDF
  • Research Article
  • Citations21

Reconfigurable Binary Neural Network Accelerator with Adaptive Parallelism Scheme

  • Jan 20, 2021
  • Electronics
  • Jaechan Cho +3
  • Conference Article
  • Citations34

VLSI implementation of deep neural networks using integral stochastic computing

  • Sep 01, 2016
  • Arash Ardakani +4
  • Research Article
  • Citations26

FPGA-based implementation of deep neural network using stochastic computing

  • Mar 05, 2023
  • Applied Soft Computing
  • Maedeh Nobari +1
  • Conference Article
  • Citations2

Circuit-level Design and Evaluation of STT-MRAM based Binary Winner-Takes-All Network for Image Recognition

  • Apr 01, 2018
  • C Wang +5
  • Conference Article
  • Citations10

Efficiency-driven Hardware Optimization for Adversarially Robust Neural Networks

  • Feb 01, 2021
  • Abhiroop Bhattacharjee +2
  • Conference Article
  • Citations16

Examining and Mitigating the Impact of Crossbar Non-idealities for Accurate Implementation of Sparse Deep Neural Networks

  • Mar 14, 2022
  • Abhiroop Bhattacharjee +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.