PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory

Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access memory (ReRAM) has showed its potential to be used for main memory. Moreover, with its crossbar array structure, ReRAM can perform matrix-vector multiplication efficiently, and has been widely studied to accelerate neural network (NN) applications. In this work, we propose a novel PIM architecture, called PRIME, to accelerate NN applications in ReRAM based main memory. In PRIME, a portion of ReRAM crossbar arrays can be configured as accelerators for NN applications or as normal memory for a larger memory space. We provide microarchitecture and circuit designs to enable the morphable functions with an insignificant area overhead. We also design a software/hardware interface for software developers to implement various NNs on PRIME. Benefiting from both the PIM architecture and the efficiency of using ReRAM for NN computation, PRIME distinguishes itself from prior work on NN acceleration, with significant performance improvement and energy saving. Our experimental results show that, compared with a state-of-the-art neural processing unit design, PRIME improves the performance by ~2360× and the energy consumption by ~895×, across the evaluated machine learning benchmarks.

The training of neural network (NN) is usually time-consuming and resource intensive. Memristor has shown its potential in computation of NN. Especially for the metal-oxide resistive random access memory (RRAM), its crossbar structure and multi-bit characteristic can perform the matrix-vector product in high precision, which is the most common operation of NN. However, there exist two challenges on realizing the training of NN. Firstly, the current architecture can only support the inference phase of training and cannot perform the backpropagation (BP), the weights update of NN. Secondly, the training of NN requires enormous iterations and constantly updates the weights to reach the convergence, which leads to large energy consumption because of lots of write and read operations. In this work, we propose a novel architecture, TIME, and peripheral circuit designs to enable the training of NN in RRAM. TIME supports the BP and the weights update while maximizing the reuse of peripheral circuits for the inference operation on RRAM. Meanwhile, a variability-free tuning scheme and gradually-write circuits are designed to reduce the cost of tuning RRAM. We explore the performance of both SL (supervised learning) and DRL (deep reinforcement learning) in TIME, and a specific mapping method of DRL is also introduced to further improve the energy efficiency. Experimental results show that, in SL, TIME can achieve 5.3× higher energy efficiency on average compared with the most powerful application-specific integrated circuits (ASIC) in the literature. In DRL, TIME can perform averagely 126× higher than GPU in energy efficiency. If the cost of tuning RRAM can be further reduced, TIME have the potential of boosting the energy efficiency by 2 orders of magnitude compared with ASIC.

Processing-in-memory (PIM) exploits massive parallelism with high energy efficiency and becomes a promising solution to the von Neumann bottleneck. Recently, the emerging metal-oxide resistive random access memory (RRAM) shows its potential to construct a PIM architecture, because several stateful logic operations, e.g., IMP and NOR, can be executed in an RRAM crossbar in parallel. Previous synthesis flows focus on improving latency with stateful logic operations, but they ignore that the memory should be used primarily for storage. i.e., most of the area in the crossbar is used for computation but not storage. In this situation, storage and computation still have to be separated into different crossbars, which leads to considerable data transfer overhead and limited parallelism. In this work, we define the ratio of storage in a crossbar as area utilization. We aim to improve the area utilization without throughput loss by proposing STAR, a novel synthesis flow for the stateful logic. We present two optimization strategies to reduce the computation area in STAR. First, we reduce the area for redundant inputs. For the shared constants among different rows (or columns), we encode them as immediate values into the control signals without writing them into the crossbar at runtime. For the other inputs, we only store one copy of them in the crossbar. Second, we reduce the area for intermediate variables by reusing invalid cells. And we design a scheduling algorithm to find a computation sequence with the minimal variable erasing cycles. Invalid primary inputs can also be erased in this algorithm. Furthermore, we present a case study of the image convolution to demonstrate the effectiveness of STAR. Experimental evaluation shows that STAR achieves 33.03% more area utilization and a 1.43x throughput compared to SIMPLER, the state-of-the-art stateful logic synthesis flow. Our image convolution implementation also provides 78.36% more area utilization and a 1.48x throughput compared with IMAGING, the state-of-the-art stateful logic-based image processing accelerator.

Resistive Random Access Memory (ReRAM) is promising to be employed as high density storage-class memory due to its crossbar array and Triple-Level Cell (TLC) structures. However, TLC crossbar ReRAM suffers from high write latency and energy due to three unique challenges: (1) The crossbar array structure incurs IR drop issues. (2) The TLC structure requires iterative program-and-verify (P&V) procedure. (3) The resistance drift problem needs short interval scrub to avoid uncorrectable soft errors. In this article, to overcome the challenges of TLC crossbar ReRAM, we propose an enhanced low latency and energy efficient TLC crossbar ReRAM architecture, called EnTiered-ReRAM. The proposed EnTiered-ReRAM is composed of four components, including EnTiered-crossbar design, Compression-based Incomplete Data Mapping (CIDM), Compression-based Flip Scheme (CFS), and Compression-based Error Correction Code (CECC). Specifically, based on the observation that our previously proposed Tiered-crossbar design still suffers from large IR drops along bitlines in the far segments due to the long length of bitlines, EnTiered-crossbar partitions each crossbar array into two halves along bitlines, and then splits each bitline of the half crossbar array into the near and far segments by an isolation transistor, which thoroughly mitigates the IR drop issues. Then we use our previously proposed CIDM and CFS in the near and far segments of EnTiered-crossbar arrays to further decreases the write latency and energy. In addition, CECC is deeply coupled with CIDM and CFS. CECC dynamically employs the most appropriate ECC capability according to the remaining space of each cache line after CIDM or CFS encoding, which effectively improves the scrub interval and performance/energy with insignificant space overhead. The evaluation results show that, compared with an aggressive baseline, EnTiered-ReRAM can improve the system performance by 56.3% and reduce the energy consumption by 60.6% on average.

Emerging resistive random-access memory (ReRAM) based processing-in-memory (PIM) accelerators have been increasingly explored in recent years because they can efficiently perform in-situ matrix-vector multiplication (MVM) operations involved in a wide spectrum of artificial neural networks. However, there remain significant challenges to apply existing ReRAM-based PIM accelerators to the most popular Transformer neural networks. Since Transformers involve a series of matrix-matrix multiplication (MatMul) operations with data dependencies, they should write intermediate results of MatMuls to ReRAM crossbar arrays for further processing. Conventional ReRAM-based PIM accelerators often suffer from high latency of ReRAM writes and intra-layer pipeline stalls. In this paper, we propose ReCAT, a ReRAM-based PIM accelerator designed particularly for Transformers. ReCAT exploits transimpedance amplifiers (TIAs) to cascade a pair of crossbar arrays for MatMul operations involved in the self-attention mechanism. The intermediate result of a MatMul generated by one crossbar array can be directly mapped to another crossbar array, avoiding costly analog-to-digital conversions. In this way, ReCAT allows MVM operations to overlap with the corresponding data mapping, hiding the high latency of ReRAM writes. Furthermore, we propose an analog-to-digital converter (ADC) virtualization scheme to dynamically share scarce ADCs among a group of crossbar arrays, and thus significantly improve the utilization of ADCs to eliminate the performance bottleneck of MVM operations. Experimental results show that ReCAT achieves 207.3×, 2.11×, and 3.06× performance improvement on average compared with other Transformer acceleration solutions—GPUs, ReBert, and ReTransformer, respectively.

Many modern workloads, such as neural networks, databases, and graph processing, are fundamentally memory-bound. For such workloads, the data movement between main memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads is insufficient to amortize the cost of main memory access. Fundamentally addressing this data movement bottleneck requires a paradigm where the memory system assumes an active role in computing by integrating processing capabilities. This paradigm is known as processing-in-memory (PIM). Recent research explores different forms of PIM architectures, motivated by the emergence of new 3D-stacked memory technologies that integrate memory with a logic layer where processing elements can be easily placed. Past works evaluate these architectures in simulation or, at best, with simplified hardware prototypes. In contrast, the UPMEM company has designed and manufactured the first publicly-available real-world PIM architecture. The UPMEM PIM architecture combines traditional DRAM memory arrays with general-purpose in- order cores, called DRAM Processing Units (DPUs), integrated in the same chip. This paper provides the first comprehensive analysis of the first publicly-available real-world PIM architecture. We make two key contributions. First, we conduct an experimental characterization of the UPMEM-based PIM system using microbenchmarks to assess various architecture limits such as compute throughput and memory bandwidth, yielding new insights. Second, we present PrIM (Processing-In-Memory benchmarks), a benchmark suite of 16 workloads from different application domains (e.g., dense/sparse linear algebra, databases, data analytics, graph processing, neural networks, bioinformatics, image processing), which we identify as memory-bound. We evaluate the performance and ...

We present a recurrent neural network (RNN) accelerator design with resistive random-access memory (ReRAM)-based processing-in-memory (PIM) architecture. Distinguished from prior ReRAM-based convolutional neural network accelerators, we redesign the system to make it suitable for RNN acceleration. We measure the system throughput and energy efficiency with the detailed circuit and device characterization. Reprogrammability is enabled with our design, and an RNN friendly pipeline is employed to increase the system throughput. We observe that on average the proposed system achieves 79× improvement of computing efficiency compared with graphics processing unit baseline. Our simulation also indicates that to maintain high accuracy and computing efficiency, the read noise standard deviation should be less than 0.2, the device resistance should be at least 1 MQ, and the device writes latency should be minimized.

Resource-constrained computing devices such as those used in IoTs require low-power, high performance, and small size to be enabled to operate efficiently. Resistive random access memory (ReRAM) is a promising technology for building novel in-memory computing architectures, due to its ability to perform storage and computation using the same physical element with low energy and high density. ReRAMbased search engines and neural network (NN) accelerators have grown significantly especially for IoT devices. In this paper, we propose a memristor-based voltage-resistance xnor (VR-XNOR) cell. The advantages of this cell are demonstrated through building a reconfigurable content-addressable memory (CAM) architecture that can support binary-CAM (BCAM) and ternary-CAM (TCAM) and enable approximate search operations. Moreover, the memristor-based VR-XNOR cell is utilized for binarized convolutional neural networks (CNN) with focus on the convolution operation. This is achieved by replacing the convolution module with XNOR-based filter banks. Simulations of the proposed architectures for search engine and feature extraction were carried out using VTEAM model in Cadence Virtuoso Analog Design Environment. The proposed filter bank architecture achieves a 1-ns extraction cycle time over N filters and produces multiple output feature maps in a single processing cycle. The filter uses two memristor devices to realize each XNOR gate and has shown a significant reduction in the number of multiply-add operations.

Resistive random access memory (ReRAM) has been proven capable to efficiently perform in-situ matrix-vector computations in convolutional neural network (CNN) processing. The computations are often conducted on multi-level cell (MLC) that have limited precision and hence, show significant vulnerability to noises. The binarized neural network (BNN) is a hardware-friendly model that can dramatically reduce the computation and storage overheads. However, XNOR, which is the key operation in BNNs, cannot be directly computed in-situ in ReRAM because of its nonlinear behavior. To enable real in-situ processing of BNNs in ReRAM, we modified the BNN algorithm to enable direct computation of XNOR, POPCOUNT and POOL based on ReRAM cells. We also proposed the complementary resistive cell (CRC) design to efficiently conduct XNOR operations and optimized the pipeline design with decoupled buffer and computation stages. Our results show our scheme, namely, ReBNN, improves the system performance by $$25.36\times$$ and the energy efficiency by $$4.26\times$$ compared to conventional ReRAM based accelerator, and ensures a throughput higher than state-of-the-art BNN accelerators. The correctness of the modified algorithm is also validated.

The modern-day computing technologies are continuously undergoing a rapid changing landscape; thus, the demands of new memory types are growing that will be fast, energy efficient and durable. The limited scaling capabilities of the conventional memory technologies are pushing the limits of data-intense applications beyond the scope of silicon-based complementary metal oxide semiconductors (CMOS). Resistive random access memory (RRAM) is one of the most suitable emerging memory technologies candidates that have demonstrated potential to replace state-of-the-art integrated electronic devices for advanced computing and digital and analog circuit applications including neuromorphic networks. RRAM has grown in prominence in the recent years due to its simple structure, long retention, high operating speed, ultra-low-power operation capabilities, ability to scale to lower dimensions without affecting the device performance and the possibility of three-dimensional integration for high-density applications. Over the past few years, research has shown RRAM as one of the most suitable candidates for designing efficient, intelligent and secure computing system in the post-CMOS era. In this manuscript, the journey and the device engineering of RRAM with a special focus on the resistive switching mechanism are detailed. This review also focuses on the RRAM based on two-dimensional (2D) materials, as 2D materials offer unique electrical, chemical, mechanical and physical properties owing to their ultrathin, flexible and multilayer structure. Finally, the applications of RRAM in the field of neuromorphic computing are presented.

As the demand for artificial intelligence continues to grow, there is a need for specialized hardware that can efficiently process and power AI algorithms. This is where VLSI (Very Large-Scale Integration) design comes into play. By integrating millions, or even billions, of transistors onto a single chip, VLSI design enables the creation of powerful AI-integrated circuits. However, designing hardware specifically for artificial intelligence poses its own set of challenges. The complexity of AI algorithms and the need for parallel processing capabilities demand innovative solutions. VLSI designers must carefully consider factors such as power consumption, heat dissipation, and the integration of specialized neural network accelerators. In addition to these considerations, VLSI designers must also address the scaling limitations of semiconductor technology. As AI algorithms become more complex, the demand for advanced VLSI architecture increases. Designers must find ways to enhance transistor density and performance while overcoming the limitations imposed by the physical laws governing semiconductor manufacturing. In-Memory Computing (IMC) is an emerging paradigm designed to overcome the limitations of the traditional Von Neumann architecture, particularly for energy-efficient AI applications. Traditional computing systems suffer from the "memory wall"—a bottleneck caused by the constant transfer of data between the CPU and memory. This leads to inefficiencies in latency, bandwidth, and energy consumption, especially for AI tasks which involve huge volumes of matrix operations and data movement. Keywords: Very Large-Scale Integration (VLSI), Artificial Intelligence (AI), In-memory computing (IMC), Neuromorphic, multiply-and-accumulate (MAC), RRAM (Resistive RAM), MRAM (Magnetoresistive RAM), PCM (Phase-Change Memory), FeFET (Ferroelectric FET), spiking neural networks (SNNs)

Among several types of emerging memory technologies, memristor-based nonvolatile resistive RAM (ReRAM) is currently being investigated as a promising candidate to potentially replace the popular Flash memories, and even other conventional memories such as SRAM and DRAM. At the architectural level, crossbar cell array structure is considered one of the best ways to implement memristor-based ReRAM. This chapter presents an overview of promising ReRAM technologies, their potential benefits, and the key research challenges, with a focus on reduction/oxidation (Redox)-based RAM. It briefly describes the basic operation principles of memristive memory cells, and presents the memristor-based crossbar memory architecture to finally focus on the serious negative impact of the current sneak-paths. Then, two possible methodologies are explored as means to deal with the sneak-path problem, concerning (i) novel storage cell structures, and (ii) modifications in the memory architecture. More specifically, (i) anti-parallel memristive switches are studied as potential cross-point elements in ReRAM arrays, in comparison with anti-serial (complementary) memristive switches. A comprehensive and comparative presentation between them is provided, while commenting on their overall performance and the most appropriate switching characteristics that the structural memristors should have, in order to better fit to memory applications. Moreover, (ii) five alternative architectures (topologies) for passive crossbar ReRAM are presented, which are based on the introduction of a certain percentage of insulating nodes spread out inside the array according to specific distribution patterns. Both approaches enable crossbar memory arrays without select devices, thus they simplify the array fabrication process and could be well-suited for future data storage applications. Finally, XbarSim, a GUI-based educational simulation tool aiming to serve students/researchers who wish to explore and study the memristive crossbar circuit architecture, is presented.

The traditional von Neumann architecture limits the increase in computing efficiency and results in massive power consumption in modern computers due to the separation of storage and processing units. The novel neuromorphic computation system, an in-memory computing architecture with low power consumption, is aimed to break the bottleneck and meet the needs of the next generation of artificial intelligence (AI) systems. Thus, it is urgent to find a memory technology to implement the neuromorphic computing nanosystem. Nowadays, the silicon-based flash memory dominates non-volatile memory market, however, it is facing challenging issues to achieve the requirements of future data storage device development due to the drawbacks, such as scaling issue, relatively slow operation speed, and high voltage for program/erase operations. The emerging resistive random-access memory (RRAM) has prompted extensive research as its simple two-terminal structure, including top electrode (TE) layer, bottom electrode (BE) layer, and an intermediate resistive switching (RS) layer. It can utilize a temporary and reversible dielectric breakdown to cause the RS phenomenon between the high resistance state (HRS) and the low resistance state (LRS). RRAM is expected to outperform conventional memory device with the advantages, notably its low-voltage operation, short programming time, great cyclic stability, and good scalability. Among the materials for RS layer, indium gallium zinc oxide (IGZO) has shown attractive prospects in abundance and high atomic diffusion property of oxygen atoms, transparency. Additionally, its electrical properties can be easily modulated by controlling the stoichiometric ratio of indium and gallium as well as oxygen potential in the sputter gas. Moreover, since the IGZO can be applied to both the thin-film transistor (TFT) channel and RS layer, it has a great potential for fully integrated transparent electronics application.In this work, we proposed amorphous transparent IGZO-based RRAMs and investigated switching behaviors of the memory cells prepared with different top electrodes. First, ITO was choosing to serve as both TE and BE to achieve high transmittance. A multi-target magnetron sputtering system was employed to deposit all three layers (TE, RS, BE layers) on glass substrate. I-V characteristics were evaluated by a semiconductor parameter analyzer, and the bipolar RS feature of our RRAM devices was demonstrated by typical butterfly curves. The optical transmission analysis was carried out via a UV-Vis spectrometer and the average transmittance was around 80% out of entire devices in the visible-light wavelength range, implying high transparency. We adjusted the oxygen partial pressure during the sputtering of IGZO to optimize the property because the oxygen vacancy concentration governs the RS performance. Electrode selection is crucial and can impact the performance of the whole device. Thus, Cu TE was chosen for our second type of device because the diffusion of Cu ions can be beneficial for the formation of the conductive filament (CF). A ~5 nm SiO2 barrier layer was employed between TE and RS layers to confine the diffusion of Cu into the RS layer. At the same time, this SiO2 inserting layer can provide an additional interfacial series resistance in the device to lower the off current, consequently, improve the on/off ratio and whole performance. Finally, an oxygen affinity metal Ti was selected as the TE for our third type of device because the concentration of the oxygen atoms can be shifted towards the Ti electrode, which provides an oxygengettering activity near the Ti metal. This process may in turn lead to the formation of a sub-stoichiometric region in the neighboring oxide that is believed to be the origin of better performance.In conclusion, the transparent amorphous IGZO-based RRAMs were established. To tune the property of RS layer, the sputtering conditions of RS were varied. To investigate the influence of TE selections on switching performance of RRAMs, we integrated a set of TE materials, and a barrier layer on IGZO-based RRAM and compared the switch characteristics. Our encouraging results clearly demonstrate that IGZO is a promising material in RRAM applications and breaking the bottleneck of current memory technologies.

Resistive random-access memories (RRAMs) operate with low power dissipation, low cost-per-bit, and high endurance, and are suitable for integration in crossbar arrays in 3D chips. These attributes make RRAM devices well-suited for a variety of applications ranging from novel processor architectures and high-density memories to neuromorphic computing and neural networks. We employ a unique suite of simulation tools to study resistance switching in oxide-based RRAM structures. We use Si-rich silica (SiOx) RRAMs as an example in this study. Silica-based RRAM technology provides an additional and unique advantage; it could be easily integrated with silicon microelectronics. Existing modeling work on RRAMs has relied heavily on classical phenomenological methods and two-dimensional (2D) modeling tools. We apply an advanced multi-scale three-dimensional (3D) simulator to investigate switching in these promising devices, and highlight their potential for low-power applications. Our physics-based electrothermal 3D simulator is very well-calibrated with experimental data, coupling self-consistently stochastic kinetic Monte Carlo descriptions of oxygen ion and electron transport, to the local electric fields and temperature. The simulator is used to demonstrate the impact of self-heating effects and also to emphasize on the necessity of 3D physical modelling to predict correctly the switching phenomenon.

Graph algorithms such as graph traversal have been gaining ever-increasing importance in the era of big data. However, graph processing on traditional architectures issues many random and irregular memory accesses, leading to a huge number of data movements and the consumption of very large amounts of energy. To minimize the waste of memory bandwidth, we investigate utilizing processing-in-memory (PIM), combined with non-volatile metal-oxide resistive random access memory (ReRAM), to improve both computation and I/O performance. We propose a new ReRAM-based processing-in-memory architecture called RPBFS, in which graph data can be persistently stored and processed in place. We study the problem of graph traversal, and we design an efficient graph traversal algorithm in RPBFS. Benefiting from low data movement overhead and high bank-level parallel computation, RPBFS shows a significant performance improvement compared with both the CPU-based and the GPU-based BFS implementations. On a suite of real-world graphs, our architecture yields a speedup in graph traversal performance of up to 33.8×, and achieves a reduction in energy over conventional systems of up to 142.8×.