• Home
  • Search
  • TxSim: Modeling Training of Deep Neural Networks on Resistive Crossbar Systems
  • Open Access IconOpen Access
  • Cite Icon52
  • https://doi.org/10.1109/tvlsi.2021.3063543Copy DOI Icon

TxSim: Modeling Training of Deep Neural Networks on Resistive Crossbar Systems

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Deep neural networks (DNNs) have gained tremendous popularity in recent years due to their ability to achieve superhuman accuracy in a wide variety of machine learning tasks. However, the compute and memory requirements of DNNs have grown rapidly, creating a need for energy-efficient hardware. Resistive crossbars have attracted significant interest in the design of the next generation of DNN accelerators due to their ability to natively execute massively parallel vector-matrix multiplications within dense memory arrays. However, crossbar-based computations face a major challenge due to device and circuit-level nonidealities, which manifest as errors in the vector-matrix multiplications and eventually degrade DNN accuracy. To address this challenge, there is a need for tools that can model the functional impact of nonidealities on DNN training and inference. Existing efforts toward this goal are either limited to inference or are too slow to be used for large-scale DNN training. We propose TxSim, a fast and customizable modeling framework to functionally evaluate DNN training on crossbar-based hardware considering the impact of nonidealities. The key features of TxSim that differentiate it from prior efforts are: 1) it comprehensively models nonidealities during all training operations (forward propagation, backward propagation, and weight update) and 2) it achieves computational efficiency by mapping crossbar evaluations to well-optimized Basic Linear Algebra Subprograms (BLAS) routines and incorporates speedup techniques to further reduce simulation time with minimal impact on accuracy. TxSim achieves 6×- 108× improvement in simulation speed over prior works, and thereby makes it feasible to evaluate the training of large-scale DNNs on crossbars. Our experiments using TxSim reveal that the accuracy degradation in DNN training due to nonidealities can be substantial (3%-36.4%) for large-scale DNNs and data sets, underscoring the need for further research in mitigation techniques. We also analyze the impact of various device and circuit-level parameters and the associated nonidealities to provide key insights that can guide the design of crossbar-based DNN training accelerators.

Similar Papers
  • Research Article
  • Citations126

RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars

  • Jun 04, 2020
  • IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
  • Shubham Jain +3
  • Research Article
  • Citations179

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

  • May 04, 2021
  • IEEE Transactions on Artificial Intelligence
  • Edgar Galván +1
  • Research Article
  • Citations59

HNPU: An Adaptive DNN Training Processor Utilizing Stochastic Dynamic Fixed-Point and Active Bit-Precision Searching

  • Mar 24, 2021
  • IEEE Journal of Solid-State Circuits
  • Donghyeon Han +6
  • Conference Article
  • Citations45

Profiling DNN Workloads on a Volta-based DGX-1 System

  • Sep 01, 2018
  • Saiful A Mojumder +7
  • Conference Article
  • Citations8

A Framework for Distributed Deep Neural Network Training with Heterogeneous Computing Platforms

  • Dec 01, 2019
  • Bontak Gu +3
  • Research Article
  • Citations7

A convergence analysis of Nesterov’s accelerated gradient method in training deep linear neural networks

  • Sep 05, 2022
  • Information Sciences
  • Xin Liu +2
  • Research Article
  • Citations6

A Guessing Entropy-Based Framework for Deep Learning-Assisted Side-Channel Analysis

  • Jan 01, 2023
  • IEEE Transactions on Information Forensics and Security
  • Ziyue Zhang +2
  • Conference Article
  • Citations2

FeFET-based Process-in-Memory Architecture for Low-Power DNN Training

  • Nov 08, 2021
  • Farzaneh Zokaee +2
  • Conference Article
  • Citations3

ADA-GP: Accelerating DNN Training By Adaptive Gradient Prediction

  • Oct 28, 2023
  • Vahid Janfaza +3
  • Conference Article
  • Citations4

MalleTrain: Deep Neural Networks Training on Unfillable Supercomputer Nodes

  • May 07, 2024
  • Xiaolong Ma +6
  • Research Article
  • Citations1

STADIA: Photonic Stochastic Gradient Descent for Neural Network Accelerators

  • Sep 09, 2023
  • ACM Transactions on Embedded Computing Systems
  • Chengpeng Xia +3
  • Conference Article
  • Citations3

AccDP: Accelerated Data-Parallel Distributed DNN Training for Modern GPU-Based HPC Clusters

  • Dec 01, 2022
  • Nawras Alnaasan +4
  • Conference Article
  • Citations66

AccPar: Tensor Partitioning for Heterogeneous Deep Learning Accelerators

  • Feb 01, 2020
  • Linghao Song +5
  • Research Article
  • Citations23

PipePar: Enabling fast DNN pipeline parallel training in heterogeneous GPU clusters

  • Aug 04, 2023
  • Neurocomputing
  • Jinghui Zhang +6
  • Research Article
  • Citations24

ApproxTrain: Fast Simulation of Approximate Multipliers for DNN Training and Inference

  • Nov 01, 2023
  • IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
  • Jing Gong +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.