CNNWire

Resistive random access memory (ReRAM) has been proven capable to efficiently perform in-situ matrix-vector computations in convolutional neural network (CNN) processing. The computations are often conducted on multi-level cell (MLC) that have limited precision and hence, show significant vulnerability to noises. The binarized neural network (BNN) is a hardware-friendly model that can dramatically reduce the computation and storage overheads. However, XNOR, which is the key operation in BNNs, cannot be directly computed in-situ in ReRAM because of its nonlinear behavior. To enable real in-situ processing of BNNs in ReRAM, we modified the BNN algorithm to enable direct computation of XNOR, POPCOUNT and POOL based on ReRAM cells. We also proposed the complementary resistive cell (CRC) design to efficiently conduct XNOR operations and optimized the pipeline design with decoupled buffer and computation stages. Our results show our scheme, namely, ReBNN, improves the system performance by $$25.36\times$$ and the energy efficiency by $$4.26\times$$ compared to conventional ReRAM based accelerator, and ensures a throughput higher than state-of-the-art BNN accelerators. The correctness of the modified algorithm is also validated.

Power-efficient data processing subsystems performing millions of complex concurrent arithmetic operations per second form part of today’s essential solution required to meet the growing demand of edge computing applications, given the volume of data collected by real-time Internet-Of-Things (IoT) sensors. Adding to it, the in-memory computation designed as memory and processing elements on a single wafer has enabled promising performance improvement in terms of computational power savings by avoiding the memory wall created while accessing the memory array. The Resistive RAM (RRAM), with its simple metal-insulator-metal (MIM) structure, proves to be a very appealing candidate for in-memory computation given its ultralow switching power and its Complementary Metal Oxide Semiconductor (CMOS) process fabrication compatibility. However, despite all advantages, the resistive switching phenomenon in RRAM has an inherent stochastic variability. On the algorithmic side, convolution neural networks (CNN) have gained popularity in image classification applications, and the network’s architecture is memory-intense in nature for memorizing the trained weights. Hence, an RRAM-based CNN system will pave way for a power-efficient image classification system on the edge. Accounting however for the inherent variability in RRAM (inter-device and intra-device), the accuracy of CNN’s prediction is surely expected to drop. This motivates us to quantify the impact of RRAM variability on the CNN trained weights and classification accuracy (prediction loss). In this study, we have constructed a Look-Up-Table (LUT) based model for encoding wide current compliance (2μA to 250μA) 65nm CMOS 1T1R OxRAM’s (TiN/HfO2/Hf/TiN) resistive variability into CNN’s trained weight in a digital regime. The RRAM resistance encoded trained weights are in turn used here to simulate the two extreme CNN architectures, namely, Fully Serial System (FSS) and Fully Parallel System (FPS). The architectures’ prediction variability trends are quantified given its current compliance, RRAM resistive variability, CNN’s convolution matrix sizes (5×5, 3×3, 1×1, and 1×1 max pool), the total number of layers in the CNN as well as the input image pixel size.

Convolutional Neural Network (CNN) accelerators are popular specialized platforms for efficient CNN processing. As semiconductor manufacturing technology scales down to nano scale, process variation dramatically affects the chip's quality. Process variation causes delay variation within the chip due to transistor parameter differences. CNN accelerators adopt a large number of Processing Elements (PEs) for parallel computing, which are highly susceptible to process variation effects. Fast CNN processing desires consistent performance among PEs, otherwise the processing speed is limited by the slowest PE within the chip. In this work, we first quantitatively model and analyze the impact of process variation on CNN accelerator's operating frequency. We further analyze the utilization of CNN accelerator and the characteristics of CNN models. We then leverage the PE underutilization to propose a sub-matrix reformation mechanism and leverage the pixel similarity of images to propose a weight transfer technique. Both techniques are able to tolerate the low-frequency PEs, and achieve performance improvement at chip level. Evaluation results show our techniques are able to achieve significant processing speed improvement with negligible accuracy loss.

Convolutional Neural Networks (CNNs) have achieved excellent performance on various artificial intelligence (AI) applications, while a higher demand on energy efficiency is required for future AI. Resistive Random-Access Memory (RRAM)-based computing system provides a promising solution to energy-efficient neural network training. However, it's difficult to support high-precision CNN in RRAM-based hardware systems. Firstly, multi-bit digital-analog interfaces will take up most energy overhead of the whole system. Secondly, it's difficult to write the RRAM to expected resistance states accurately; only low-precision numbers can be represented. To enable CNN training based on RRAM, we propose a low-bitwidth CNN training method, using low-bitwidth convolution outputs (CO), activations (A), weights (W) and gradients (G) to train CNN models based on RRAM. Furthermore, we design a system to implement the training algorithms. We explore the accuracy under different bitwidth combinations of (A, CO, W, G), and propose a practical tradeoff between accuracy and energy overhead. Our experiments demonstrate that the proposed system perform well on low-bitwidth CNN training tasks. For example, training LeNet-5 with 4-bit convolution outputs, 4-bit weights, 4-bit activations and 4-bit gradients on MNIST can still achieve 97.67% accuracy. Moreover, the proposed system can achieve 23.0X higher energy efficiency than GPU when processing the training task of LeNet-5, and 4.4X higher energy efficiency when processing the training task of ResNet-20.

Resource-constrained computing devices such as those used in IoTs require low-power, high performance, and small size to be enabled to operate efficiently. Resistive random access memory (ReRAM) is a promising technology for building novel in-memory computing architectures, due to its ability to perform storage and computation using the same physical element with low energy and high density. ReRAMbased search engines and neural network (NN) accelerators have grown significantly especially for IoT devices. In this paper, we propose a memristor-based voltage-resistance xnor (VR-XNOR) cell. The advantages of this cell are demonstrated through building a reconfigurable content-addressable memory (CAM) architecture that can support binary-CAM (BCAM) and ternary-CAM (TCAM) and enable approximate search operations. Moreover, the memristor-based VR-XNOR cell is utilized for binarized convolutional neural networks (CNN) with focus on the convolution operation. This is achieved by replacing the convolution module with XNOR-based filter banks. Simulations of the proposed architectures for search engine and feature extraction were carried out using VTEAM model in Cadence Virtuoso Analog Design Environment. The proposed filter bank architecture achieves a 1-ns extraction cycle time over N filters and produces multiple output feature maps in a single processing cycle. The filter uses two memristor devices to realize each XNOR gate and has shown a significant reduction in the number of multiply-add operations.

We empirically evaluate an undervolting technique, i.e., underscaling the\ncircuit supply voltage below the nominal level, to improve the power-efficiency\nof Convolutional Neural Network (CNN) accelerators mapped to Field Programmable\nGate Arrays (FPGAs). Undervolting below a safe voltage level can lead to timing\nfaults due to excessive circuit latency increase. We evaluate the\nreliability-power trade-off for such accelerators. Specifically, we\nexperimentally study the reduced-voltage operation of multiple components of\nreal FPGAs, characterize the corresponding reliability behavior of CNN\naccelerators, propose techniques to minimize the drawbacks of reduced-voltage\noperation, and combine undervolting with architectural CNN optimization\ntechniques, i.e., quantization and pruning. We investigate the effect of\nenvironmental temperature on the reliability-power trade-off of such\naccelerators. We perform experiments on three identical samples of modern\nXilinx ZCU102 FPGA platforms with five state-of-the-art image classification\nCNN benchmarks. This approach allows us to study the effects of our\nundervolting technique for both software and hardware variability. We achieve\nmore than 3X power-efficiency (GOPs/W) gain via undervolting. 2.6X of this gain\nis the result of eliminating the voltage guardband region, i.e., the safe\nvoltage region below the nominal level that is set by FPGA vendor to ensure\ncorrect functionality in worst-case environmental and circuit conditions. 43%\nof the power-efficiency gain is due to further undervolting below the\nguardband, which comes at the cost of accuracy loss in the CNN accelerator. We\nevaluate an effective frequency underscaling technique that prevents this\naccuracy loss, and find that it reduces the power-efficiency gain from 43% to\n25%.\n

Convolutional Neural Networks (CNNs) play a vital role in machine learning. CNNs are typically both computing and memory intensive. Emerging resistive random-access memories (RRAMs) and RRAM crossbars have demonstrated great potentials in boosting the performance and energy efficiency of CNNs. Compared with small crossbars, large crossbars show better energy efficiency with less interface overhead. However, conventional workload mapping methods for small crossbars cannot make full use of the computation ability of large crossbars. In this paper, we propose an Overlapped Mapping Method (OMM) and MIxed Size Crossbar based RRAM CNN Accelerator (MISCA) to solve this problem. MISCA with OMM can reduce the energy consumption caused by the interface circuits, and improve the parallelism of computation by leveraging the idle RRAM cells in crossbars. The simulation results show that MISCA with OMM can achieve 2.7× speedup, 30% utilization rate improvement, and 1.2× energy efficiency improvement on average compared with fixed size crossbars based accelerator using the conventional mapping method. In comparison with GPU platform, MISCA with OMM can perform 490.4× higher on average in energy efficiency and 20× higher on average in speedup. Compared with PRIME, an existing RRAM based accelerator, MISCA has 26.4× speedup and 1.65× energy efficiency improvement.

We present a recurrent neural network (RNN) accelerator design with resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architecture. Distinguished from prior ReRAM-based convolutional neural network accelerators, we redesign the system to make it suitable for RNN acceleration. We measure the system throughput and energy efficiency with the detailed circuit and device characterization. Reprogrammability is enabled with our design, and an RNN friendly pipeline is employed to increase the system throughput. We observe that on average the proposed system achieves 79× improvement of computing efficiency compared with graphics processing unit baseline. Our simulation also indicates that to maintain high accuracy and computing efficiency, the read noise standard deviation should be less than 0.2, the device resistance should be at least 1 MQ, and the device writes latency should be minimized.

Deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance in classification, natural language processing (NLP), and regression tasks. However, there is still a great gap between DCNNs and the human brain in terms of computation efficiency. Inspired by neural synaptic plasticity and stochastic computing (SC), we propose neural synaptic plasticity-inspired computing (NSPC) to simulate the human brain's neural network activity for inference tasks with simple logic gates. The multiplication and accumulation (MAC) is transformed by the wire connectivity in NSPC, which only requires bundles of wires and small width adders. To this end, the NSPC imitates the structure of neural synaptic plasticity from a circuit wires connection perspective. Furthermore, from the principle of NSPC, we use a data mapping method to convert the convolution operations to matrix multiplications. Based on the methodology of NSPC, fully-pipelined and low latency architecture is designed. The proposed NSPC accelerator exhibits high hardware efficiency while maintaining a comparable network accuracy level. The NSPC based DCNN accelerator (NSPC-CNN) processes DCNN at 1.5625M images/s with a power dissipation of 15.42 W and an area of 36.4 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> . The NSPC based deep neural network (DNN) accelerator (NSPC-DNN) that implements three fully connected layers DNN consumes only 6.6 mm2 area and 2.93 W power, and achieves a throughput of 400M images/s. Compared with conventional fixed-point implementations, the NSPC-CNN achieves 2.77× area efficiency, 2.25× power efficiency; the proposed NSPC-DNN exhibits 2.31× area efficiency and 2.09× power efficiency.

Deep neural networks (DNNs) are tremendously applied in artificial intelligence field. While the performance of DNNs is continuously improved by more complicated and deeper structures, the feasibility of deployment on embedded system remains as a critical problem. Neuromorphic system designs based on resistive random-access memory (ReRAM) provide an opportunity for power-efficient DNN employment. However, it encounters the challenge of limited programming resolution. A quantized training method is proposed in this paper to enhance the performance of neuromorphic systems based on ReRAM. Different from previous methods in which a dedicated regularization term is used in loss function to constrain parameter distribution, our quantized training method deals with training and quantization at the same time to alleviate the impact of limited parameter precision. Models with discrete parameters obtained after training can be directly mapped onto ReRAM devices. We implement experiments on image recognition tasks using multi-layer perceptron (MLP) and convolution neural network (CNN). The results verify that the quantized training method can approximate the accuracy of full-precision training, e.g., a two-layer MLP based on binary ReRAM only decreases the classification accuracy by 0.25% for MNIST dataset. In addition, we carefully investigate and present the importance of layer size under the ReRAM's low programming resolution, the different parameter resolution demands for convolution layer and fully connected layer, and system robustness to ReRAM variations after quantized training. The codes are available at https://github.com/qingyangqing/quantized-rram-net.

Recently neural network accelerators have grown into prominence with significant power and performance efficiency improvements over CPU and GPU. In this paper, we proposed two safety design techniques include Algorithm Based Atomic Error Checking-1 (ABAEC-1) and ABAEC-2 for a Weight Stationary (WS) Convolutional Neural Network (CNN) accelerator focusing on low latency and low overhead error detection and correction with no performance degradation. The proposed design techniques not only detect the errors on-the-fly but also perform error diagnosis to localize the errors to a Processing Element (PE) for on-line fault management and recovery. We applied the design techniques on an industry quality CNN accelerator and demonstrated that we could achieve the required Diagnostic Coverage (DC) goal with minimal area and power overhead for selected configurations. Furthermore, we discussed methods to extend the proposed techniques to other dataflow architecture.

Convolutional neural network (CNN) accelerators have achieved great success from cloud to edge scenarios. However, given the trend towards even larger and deeper neural network models, it remains a challenging problem to efficiently process these CNNs especially on edge devices with limited energy budget. Accordingly, reducing the energy consumption is of paramount importance for sustainable CNN accelerators. In this paper, we propose AdaPrune, a novel pruning technique that reduces model size and computation to achieve performance improvement and energy savings for CNN accelerators. Unlike previous pruning techniques that sacrifice either computational regularity or accuracy, AdaPrune maintains both by customizing CNN pruning for the underlying accelerators to maximally leverage the sparsity benefits. AdaPrune consists of two techniques: input channel group pruning and output channel group pruning. By analyzing the weight fetching patterns of sparse CNN accelerators, AdaPrune adaptively switches between the two techniques to guarantee that the zeros are evenly distributed in each fetching group. In doing so, the pruned network structure preserves customized computational regularity for the underlying accelerators, thereby boosting the performance and energy efficiency. We evaluate AdaPrune on three sparse CNN accelerators with different spatial tiling strategies. The experimental results show that AdaPrune achieves up to 1.6&#x00D7; performance speedup, and 1.5&#x00D7; energy savings compared to unstructured pruning.

Realization of spiking neural network (SNN) hardware with high energy efficiency and high integration may provide a promising solution to data processing challenges in future internet of things (IoT) and artificial intelligence (AI). Recently, design of multi-core reconfigurable SNN chip based on resistive random-access memory (RRAM) is drawing great attention, owing to the unique properties of RRAM, e.g., high integration density, low power consumption, and processing-in-memory (PIM). Therefore, RRAM-based SNN chip may have further improvements in integration and energy efficiency. The design of such a chip will face the following problems: significant delay in pulse transmission due to complex logic control and inter-core communication; high risk of digital, analog, and RRAM hybrid design; and non-ideal characteristics of analog circuit and RRAM. In order to effectively bridge the gap between device, circuit, algorithm, and architecture, this paper proposes a simulation model—FangTianSim, which covers analog neuron circuit, RRAM model and multi-core architecture and its accuracy is at the clock level. This model can be used to verify the functionalities, delay, and power consumption of SNN chip. This information cannot only be used to verify the rationality of the architecture but also guide the chip design. In order to map different network topologies on the chip, SNN representation format, interpreter, and instruction generator are designed. Finally, the function of FangTianSim is verified on liquid state machine (LSM), fully connected neural network (FCNN), and convolutional neural network (CNN).

Training Convolutional Neural Networks(CNNs) is both memory-and computation-intensive. The resistive random access memory (ReRAM) has shown its advantage to accelerate such tasks with high energy-efficiency. However, the ReRAM-based pipeline architecture suffers from the low utilization of computing resource, caused by the imbalanced data throughput in different pipeline stages because of the inherent down-sampling effect in CNNs and the inflexible usage of ReRAM cells. In this paper, we propose a novel ReRAM-based bidirectional pipeline architecture, named HUBPA, to accelerate the training with higher utilization of the computing resource. Two stages of the CNN training, forward and backward propagations, are scheduled in HUBPA dynamically to share the computing resource. We design an accessory control scheme for the context switch of these two tasks. We also propose an efficient algorithm to allocate computing resource for each neural network layer. Our experiment results show that, compared with state-of-the-art ReRAM pipeline architecture, HUBPA improves the performance by 1.7X and reduces the energy consumption by 1.5X, based on the current benchmarks.

Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access memory (ReRAM) has showed its potential to be used for main memory. Moreover, with its crossbar array structure, ReRAM can perform matrix-vector multiplication efficiently, and has been widely studied to accelerate neural network (NN) applications. In this work, we propose a novel PIM architecture, called PRIME, to accelerate NN applications in ReRAM based main memory. In PRIME, a portion of ReRAM crossbar arrays can be configured as accelerators for NN applications or as normal memory for a larger memory space. We provide microarchitecture and circuit designs to enable the morphable functions with an insignificant area overhead. We also design a software/hardware interface for software developers to implement various NNs on PRIME. Benefiting from both the PIM architecture and the efficiency of using ReRAM for NN computation, PRIME distinguishes itself from prior work on NN acceleration, with significant performance improvement and energy saving. Our experimental results show that, compared with a state-of-the-art neural processing unit design, PRIME improves the performance by ~2360× and the energy consumption by ~895×, across the evaluated machine learning benchmarks.