(Invited) In-Memory Computing with Memristor Circuit Primitives

Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access memory (ReRAM) has showed its potential to be used for main memory. Moreover, with its crossbar array structure, ReRAM can perform matrix-vector multiplication efficiently, and has been widely studied to accelerate neural network (NN) applications. In this work, we propose a novel PIM architecture, called PRIME, to accelerate NN applications in ReRAM based main memory. In PRIME, a portion of ReRAM crossbar arrays can be configured as accelerators for NN applications or as normal memory for a larger memory space. We provide microarchitecture and circuit designs to enable the morphable functions with an insignificant area overhead. We also design a software/hardware interface for software developers to implement various NNs on PRIME. Benefiting from both the PIM architecture and the efficiency of using ReRAM for NN computation, PRIME distinguishes itself from prior work on NN acceleration, with significant performance improvement and energy saving. Our experimental results show that, compared with a state-of-the-art neural processing unit design, PRIME improves the performance by ~2360x and the energy consumption by ~895x, across the evaluated machine learning benchmarks.

Resource-constrained computing devices such as those used in IoTs require low-power, high performance, and small size to be enabled to operate efficiently. Resistive random access memory (ReRAM) is a promising technology for building novel in-memory computing architectures, due to its ability to perform storage and computation using the same physical element with low energy and high density. ReRAMbased search engines and neural network (NN) accelerators have grown significantly especially for IoT devices. In this paper, we propose a memristor-based voltage-resistance xnor (VR-XNOR) cell. The advantages of this cell are demonstrated through building a reconfigurable content-addressable memory (CAM) architecture that can support binary-CAM (BCAM) and ternary-CAM (TCAM) and enable approximate search operations. Moreover, the memristor-based VR-XNOR cell is utilized for binarized convolutional neural networks (CNN) with focus on the convolution operation. This is achieved by replacing the convolution module with XNOR-based filter banks. Simulations of the proposed architectures for search engine and feature extraction were carried out using VTEAM model in Cadence Virtuoso Analog Design Environment. The proposed filter bank architecture achieves a 1-ns extraction cycle time over N filters and produces multiple output feature maps in a single processing cycle. The filter uses two memristor devices to realize each XNOR gate and has shown a significant reduction in the number of multiply-add operations.

Due to the voltage driven switching at low voltages combined with nonvolatility of the achieved polarization state, ferroelectric materials have a unique potential for low power nonvolatile electronic devices. The competitivity of such devices is hindered by compatibility issues of well-known ferroelectrics with established semiconductor technology. The discovery of ferroelectricity in hafnium oxide changed this situation. The natural application of nonvolatile devices is as a memory cell. Nonvolatile memory devices also built the basis for other applications like in-memory or neuromorphic computing. Three different basic ferroelectric devices can be constructed: ferroelectric capacitors, ferroelectric field effect transistors and ferroelectric tunneling junctions. In this article first the material science of the ferroelectricity in hafnium oxide will be summarized with a special focus on tailoring the switching characteristics towards different applications.The current status of nonvolatile ferroelectric memories then lays the ground for looking into applications like in-memory computing. Finally, a special focus will be given to showcase how the basic building blocks of spiking neural networks, the neuron and the synapse, can be realized and how they can be combined to realize neuromorphic computing systems. A summary, comparison with other technologies like resistive switching devices and an outlook completes the paper.

The high demand for performance and energy efficiency poses significant challenges for computing systems in recent years. The memristor-based crossbar array architecture is enthusiastically regarded as a potential competitor to traditional solutions due to its low power consumption and fast switching speed. Especially by leveraging self-rectifying memristive devices, passive crossbar arrays potentially enable high memory densities. Nonetheless, due to the lack of a switching control per cell, these passive, self-rectifying memristive crossbar arrays (srMCA) suffer from sneak path current issues that limit the range of accurate operation of the crossbar array. In this work, the sneak path current issues in the passive srMCAs based on self-rectifying bipolar and complementary switching memristive devices are comparatively analyzed. Under consideration of the worst-case scenario, three reading schemes are investigated: one wordline pull-up (OneWLPU), all wordline pull-up (AllWLPU), and floating (FL) reading schemes. As a conclusion, despite different switching dynamics, both types of self-rectifying memristive devices can efficiently suppress sneak path current in the srMCAs. In the FL reading scheme, the sneak path current flowing through the unselected reversely biased memristive cells in the srMCA can be considered as an accurate estimation for the practical sneak path current in the srMCA. By analyzing the sneak path current in the srMCAs with a size up to 64 × 64, it is demonstrated that the leakage current plays a crucial role for suppressing the sneak path current, and the sneak path current via an individual cell exhibits a continuous decrease while the accumulated total sneak path current in the unselected reverse biased region is increasing with expanding the crossbar size. The comparative study on the bipolar and complementary memristive devices based srMCAs under diverse reading schemes reveals the influence of the switching dynamics on the sneak path current effect in the srMCAs, and provides a beneficial reference and feasible solutions for the future optimization of the crossbar topology with the intention of mitigating sneak path effects.

Convolutional Neural Network (CNN) is one of the most important Deep Neural Networks (DNN) classes that helps solving many tasks related to image recognition and computer vision. Their classical implementations by using conventional CMOS technologies and digital design techniques are still considered very energy-consuming. Floating point CNN relies primarily on MAC (Multiply and ACcumulate) operation. Recently, cost-effective Bite-wise CNN based on XNOR and bit-counting operations have been considered as a possible hardware implementation candidate. However, the Von-Neumann bottleneck due to intensive data fetching between memory and the computing core limits their scalability on hardware. XNOR-BITCOUNT operations can be easily implemented by using In Memory Computing (IMC) paradigms executed on a memristive crossbar array. Among emerging memristive devices, the Spin-Orbit Torque Magnetic Random Access Memory (SOT-MRAM) offers the possibility to have a higher ON resistance that allows reducing the reading current, since all the crossbar array is read in parallel. This could contribute to a further reduction of energy consumption, paving the way for much bigger crossbar designs. This study presents a crossbar architecture based on SOT-MRAM with very low energy consumption; we study the impact of process variability on the synaptic weights and perform Monte-Carlo simulations of the overall crossbar array to evaluate the error rate. Simulation results show that this implementation has lower energy consumption with respect to other memristive solutions with 65.89 fJ per read operation. The design is also quite robust to process variations, with very low reading inaccuracies up to 10 %.

Resistive memory, also known as memristor, is an emerging potential successor to traditional CMOS charge based memories. Memristors have also recently been proposed as a promising candidate for several additional applications such as logic design, sensing, non-volatile storage, neuromorphic computing, Physically Unclonable Functions (PUFs), Content-addressable memory (CAM) and reconfigurable computing. In this paper, we explore three unique applications of memristor technology based implementations, specifically from the perspective of sensing, logic, in-memory computing and their solutions. We review solar cell health monitoring and diagnosis, describe the proposed solutions, and provide directions in memristive gas sensing and in-memory computing. For the gas sensor application, in order to determine the number of memristors to ensure a certain level of accuracy in sensitivity, a technique to optimize the sensor array based on an acceptable sensitivity variation and minimum sensitivity margin is presented. These “out-of-the-box” emerging ideas for applications of memristive devices in enhancing robustness and, at the same time, how the requirements of robust design are enabling unconventional use of the devices. To this end, the papers considers some examples of this mutual interaction.

The tunability of conductance states of various emerging nonvolatile memristive devices emulates the plasticity of biological synapses, making it promising in the hardware realization of large‐scale neuromorphic systems. The inference of the neural network can be greatly accelerated by the vector‐matrix multiplication (VMM) performed within a crossbar array of memristive devices in one step. Nevertheless, the implementation of the VMM needs complex peripheral circuits, and the complexity further increases as non‐idealities of memristive devices prevent precise conductance tuning (especially for the online training) and largely degrade the performance of the deep neural networks (DNNs). Herein, an efficient online training method of the memristive deep belief net (DBN) is presented. The proposed memristive DBN uses stochastically binarized activations, reducing the complexity of peripheral circuits, and uses the contrastive divergence (CD)‐based gradient descent learning algorithm. The analog VMM and digital CD are performed separately in a mixed‐signal hardware arrangement, making the memristive DBN highly immune to non‐idealities of synaptic devices. The number of write operations on memristive devices is reduced by two orders of magnitude. The recognition accuracy of 95–97% can be achieved for the MNIST dataset using pulsed synaptic behaviors of various memristive synaptic devices.

In-memory computing facilitates efficient parallel computing based on the programmable memristor crossbar array. Proficient hardware image processing can be implemented by utilizing the analog vector-matrix operation with multiple memory states of the nonvolatile memristor in the crossbar array. Among various materials, 2D materials are great candidates for a switching layer of nonvolatile memristors, demonstrating low-power operation and electrical tunability through their remarkable physical and electrical properties. However, the intrinsic device-to-device (D2D) variation of memristors within the crossbar array can degrade the accuracy and performance of in-memory computing. Here, we demonstrate hardware image processing using the fabricated 2D hexagonal boron nitride-based memristor to investigate the effects of D2D variation on the hardware convolution process. The image quality is evaluated by peak-signal-to-noise ratio, structural similarity index measure, and Pratt’s figure of merit and analyzed according to D2D variations. Then, we propose a novel two-step gradual reset programming scheme to enhance the conductance uniformity of multiple states of devices. This approach can enhance the D2D variation and demonstrate the improved quality of the image processing result. We believe that this result suggests the precise tuning method to realize high-performance in-memory computing.

In-memory computing (IMC) has been proposed to overcome the von Neumann bottleneck in data-intensive applications. However, existing IMC solutions could not achieve both high parallelism and high flexibility, which limits their application in more general scenarios: As a highly parallel IMC design, the functionality of a MAC crossbar is limited to the matrix-vector multiplication; Another IMC method of logic-in-memory (LiM) is more flexible in supporting different logic functions, but has low parallelism. To improve the LiM parallelism, we are inspired by investigating how the single-instruction, multiple-data (SIMD) instruction set in conventional CPU could potentially help to expand the number of LiM operands in one cycle. The biggest challenge is the inefficiency in handling non-continuous data in parallel due to the SIMD limitation of (i) continuous address, (ii) limited cache bandwidth, and (iii) large full-resolution parallel computing overheads. This article presents GRAPHIC, the first reported in-memory SIMD architecture that solves the parallelism and irregular data access challenges in applying SIMD to LiM. GRAPHIC exploits content-addressable memory (CAM) and row-wise-accessible SRAM. By providing the in-situ, full-parallelism, and low-overhead operations of address search, cache read-compute-and-update, GRAPHIC accomplishes high-efficiency gather and aggregation with high parallelism, high energy efficiency, low latency, and low area overheads. Experiments in both continuous data access and irregular data pattern applications show an average speedup of 5x over iso-area AVX-like LiM, and 3-5x over the emerging CAM-based accelerators of CAPE and GaaS-X in advanced techniques.

Content addressable memories (CAMs) are special-purpose in-memory computing units that support parallel searches directly in memory. There is growing interest in CAMs for data-intensive applications such as machine learning, data mining, and bioinformatics, which has led to a rapidly growing CAM design space. CAM cells can be implemented exclusively by CMOS or with various non-volatile memory (NVM) devices. In addition to traditional binary and ternary CAMs (BCAMs and TCAMs), analog CAM (ACAM) and multi-bit CAM (MCAM) designs have recently been introduced, which could further improve density, and also support unique in-memory distance functions. Furthermore, aside from the widely-used exact match function, CAM-based approximate match functions, such as threshold match and best match, have been proposed to further extend the utility of CAMs to new application spaces. As the CAM design space is large, evaluating different CAM design options for a given application is both crucial and challenging. This paper presents EvaCAM, a circuit-level modeling and evaluation tool for CAMs. EvaCAM supports TCAM, ACAM, and MCAM designs implemented in either CMOS or NVMs, for both exact and approximate match functions. It also allows for the exploration of different CAM designs under various optimization targets. EvaCAM has been validated against measured data from fabricated chips and detailed SPICE simulations. A comprehensive design space exploration for CAMs is provided to illustrate the impact of various design decisions and to demonstrate the use cases of EvaCAM.

A compact quaternary cell circuit using a single MOS transistor and two magnetic tunnel junction devices (1T-2MTJ) is proposed for a high-density nonvolatile bit-serial content-addressable memory (CAM). The use of quaternary CAM-cell structure makes the search-cycle counts half, which achieves a high-speed search operation. Moreover, the power supply of a CAM word circuit is cut off whenever a mismatched cell is detected during search operation, which greatly reduces the static power dissipation. In fact, the average activation ratio of a 128-bit CAM word circuit is about 2.05 percent. The efficiency of the proposed CAM-cell structure is discussed in comparison with a conventional binary CAM-cell structure under a 0.14 um CMOS/MTJ technology.

Scalable in-memory mapping of Boolean functions in memristive crossbar array using simulated annealing

Processing-in-memory (PIM) is a promising solution to address the "memory wall" challenges for future computer systems. Prior proposed PIM architectures put additional computation logic in or near memory. The emerging metal-oxide resistive random access memory (ReRAM) has showed its potential to be used for main memory. Moreover, with its crossbar array structure, ReRAM can perform matrix-vector multiplication efficiently, and has been widely studied to accelerate neural network (NN) applications. In this work, we propose a novel PIM architecture, called PRIME, to accelerate NN applications in ReRAM based main memory. In PRIME, a portion of ReRAM crossbar arrays can be configured as accelerators for NN applications or as normal memory for a larger memory space. We provide microarchitecture and circuit designs to enable the morphable functions with an insignificant area overhead. We also design a software/hardware interface for software developers to implement various NNs on PRIME. Benefiting from both the PIM architecture and the efficiency of using ReRAM for NN computation, PRIME distinguishes itself from prior work on NN acceleration, with significant performance improvement and energy saving. Our experimental results show that, compared with a state-of-the-art neural processing unit design, PRIME improves the performance by ~2360× and the energy consumption by ~895×, across the evaluated machine learning benchmarks.

Three-dimensional (3D) stackable memory devices including nano-scaled crossbar array are central for the realization of high-density non-volatile memory electronics. However, an essential sneak path issue affecting device performance in crossbar array remains a bottleneck and a grand challenge. Therefore, a suitable bidirectional selector as a two-way switch is required to facilitate a major breakthrough in the 3D crossbar array memory devices. Here, we show the excellent selectivity of all oxide p-/n-type semiconductor-based p-n-p open-based bipolar junction transistors as selectors in crossbar memory array. We report that bidirectional nonlinear characteristics of oxide p-n-p junctions can be highly enhanced by manipulating p-/n-type oxide semiconductor characteristics. We also propose an associated Zener tunneling mechanism that explains the unique features of our p-n-p selector. Our experimental findings are further extended to confirm the profound functionality of oxide p-n-p selectors integrated with several bipolar resistive switching memory elements working as storage nodes.

Memristor-based analogue computing in memory (CIM) offers revolutionary gains in energy efficiency and computing power for data-intensive applications such as artificial intelligence. However, it typically struggles with achieving high accuracy at the same time, owing to the noise-sensitive nature of analogue computing and the non-ideal characteristics at the device and circuit levels that inevitably result in computing errors. Although progress has been made in device engineering and hardware-algorithm co-optimization to mitigate the error and parasitic effects, many of these advances inadvertently incur a hardware or energy consumption overhead, undermining the core benefits of analogue CIM. This Review dissects the computing error sources across the CIM hierarchy from the memristor device and array to the system architecture and algorithm, and evaluates the strategies to minimize those errors. We highlight the material and device innovations, array-level techniques and algorithm-architecture co-design frameworks towards high-accuracy analogue CIM. By dissecting the trade-off between computing accuracy and implementation cost, this Review draws a roadmap for translating memristor-based analogue CIM technology from proof-of-concept prototypes to large-scale deployment for accelerating next-generation artificial intelligence.