Three-Dimensional Neuromorphic Computing System With Two-Layer and Low-Variation Memristive Synapses

Among many artificial neural networks, the research on Spike Neural Network (SNN), which mimics the energy-efficient signal system in the brain, is drawing much attention. Memristor is a promising candidate as a synaptic component for hardware implementation of SNN, but several non-ideal device properties are making it challengeable. In this work, we conducted an SNN simulation by adding a device model with a non-linear weight update to test the impact on SNN performance. We found that SNN has a strong tolerance for the device non-linearity and the network can keep the accuracy high if a device meets one of the two conditions: 1. symmetric LTP and LTD curves and 2. positive non-linearity factors for both LTP and LTD. The reason was analyzed in terms of the balance between network parameters as well as the variability of weight. The results are considered to be a piece of useful prior information for the future implementation of emerging device-based neuromorphic hardware.

A spiking neural network (SNN) is considered a high-performance learning system that matches the digital circuits and presents higher efficiency due to the architecture and computation of spiking neurons. While implementing a SNN on a field-programmable gate array (FPGA), the gradient back-propagation through layers consumes a surprising number of resources. In this paper, we aim to realize an efficient architecture of SNN on the FPGA to reduce resource and power consumption. The multi-compartment leaky integrate-and-fire (MLIF) model is used to convert spike trains to the plateau potential in dendrites. We accumulate the potential in the apical dendrite during the training period. The average of this accumulative result is the dendritic plateau potential and is used to guide the updates of synaptic weights. Based on this architecture, the SNN is implemented on FPGA efficiently. In the implementation of a neuromorphic learning system, the shift multiplier (shift MUL) module and piecewise linear (PWL) algorithm are used to replace multipliers and complex nonlinear functions to match the digital circuits. The neuromorphic learning system is constructed with resources on FPGA without dataflow between on-chip and off-chip memories. Our neuromorphic learning system performs with higher resource utilization and power efficiency than previous on-chip learning systems.

Spiking neural networks (SNNs) are the basis for many energy-efficient neuromorphic hardware systems. While there has been substantial progress in SNN research, artificial SNNs still lack many capabilities of their biological counterparts. In biological neural systems, memory is a key component that enables the retention of information over a huge range of temporal scales, ranging from hundreds of milliseconds up to years. While Hebbian plasticity is believed to play a pivotal role in biological memory, it has so far been analyzed mostly in the context of pattern completion and unsupervised learning in artificial and SNNs. Here, we propose that Hebbian plasticity is fundamental for computations in biological and artificial spiking neural systems. We introduce a novel memory-augmented SNN architecture that is enriched by Hebbian synaptic plasticity. We show that Hebbian enrichment renders SNNs surprisingly versatile in terms of their computational as well as learning capabilities. It improves their abilities for out-of-distribution generalization, one-shot learning, cross-modal generative association, language processing, and reward-based learning. This suggests that powerful cognitive neuromorphic systems can be built based on this principle.

This work evaluates the performance of Neuromorphic architecture for accessing Static Random Access Memory (SRAM) in an asynchronous manner. Spike-Timing Dependent Plasticity (STDP) learning algorithm for updating the synaptic weights values in the SRAM module. Input, output, and synaptic weight values are transmitted to/from the chip using a common communication protocol based on the Address Event Representation (AER). Memory array an important block in digital system. Bassically SRAM is used for increasing speed of the processor. Which is mostly connected between the DRAM and microprocessor. SRAM serves as cache memory which stores the information and losses its data without power. We may design the SRAM using 4T, 6T, 8T....for storing one bit. Sense amplifier is used to perform read and write operation. Instead of using sense amplifier we use the neural network concept. Synapse is connected between the SRAM and neuron. The minute gap across which nerve impulses pass from one neurone to the next, at the end of a nerve fiber. Each neurone has an enlarged portion the cell body, containing the nucleus; from the body extend several process through which impulses enter from their branches. A longer process, the nerve fiber, extends outwards and carries impulse away from the cell body. This is normally unbranched except at the nerve ending. This consumes less power than the normal memory circuit.

As Moore's law ends, the conventional von Neumann computer architecture with binary-coded data representation has reached its bottleneck because of having separate computing and memory modules. In this architecture, continuous power is required due to sequential processing, making it challenging to improve efficiency further. The human brain can be regarded as the most energy-efficient computer architecture. In the brain, data is represented as small voltage pulses as spikes. That is why the neural units are called spiking neural networks (SNN). In SNNs, energy is required only when there is a spike, making it more energy efficient than the von-Neumann computer architecture. Therefore, in recent decades, researchers in this field have been interested in mimicking the data processing of the human brain in electronic circuits. In biological SNNs, data is propagated from one neuron to another via a synapse. When there is an incoming spike, the chemical weight changes in the synapse, and the next neuron receives the signal. In the electronic neural network, a memristor, a two-terminal nonvolatile memory element, can emulate the function of a synapse by changing its conductance while receiving a pulse. The most common learning rule in the spiking neural network is Spike-Timing-Dependent Plasticity (STDP), which refers to the change of synaptic weight with respect to the time difference between pre- and post-synaptic neural spikes. Although electronic SNNs comprised of memristors with the STDP learning rule have shown promising performance in various event-driven tasks, circuit complexity limitations and a lack of third-order parametric inclusion exist. A solution can be using a simple circuit element to modulate the memristor response.

A low power CMOS Static Random Access Memory (SRAM) based Field Programmable Gate Arrays (FPGA) architecture is being presented in this paper. The architecture presented here is based on CMOS logic and CMOS SRAMs that are used for on-chip dynamic reconfiguration. This architecture employs the fast and low-power SRAM blocks that are based on 10T SRAM cells. These blocks are employed in fast access of the configuration bits by using the shadow SRAM technique. The dynamic reconfiguration delay is being hidden behind the computation delay through the use of shadow SRAM cells. The combined effect of both the SRAM memory cells and the shadow SRAM scheme enables to support in reducing the delay and also to achieve reduced power consumption. Experimental results show reduced delay of about 8.035ns and power consumption of about 0.015W for the 10T SRAM memory cell with an overhead in area, relative to 4T and 6T SRAM cells. Also, the experimental results include the values of delay of about 8.979ns and power consumption of about 0.052W, achieved for the LB of FPGA architecture which employs CMOS SRAMs using the 10T SRAM memory cells in it.

In this paper, we propose a system for file classification in large data sets based on spiking neural networks (SNNs). File information contained in key-value metadata pairs is mapped by a novel correlative temporal encoding scheme to spike patterns that are input to an SNN. The correlation between input spike patterns is determined by a file similarity measure. Unsupervised training of such networks using spike-timing-dependent plasticity (STDP) is addressed first. Then, supervised SNN training is considered by backpropagation of an error signal that is obtained by comparing the spike pattern at the output neurons with a target pattern representing the desired class. The classification accuracy is measured for various publicly available data sets with tens of thousands of elements, and compared with other learning algorithms, including logistic regression and support-vector machines. Simulation results indicate that the proposed SNN-based system using memristive synapses may represent a valid alternative to classical machine learning algorithms for inference tasks, especially in environments with asynchronous ingest of input data and limited resources.

Neuromorphic computing is an unconventional computing scheme that executes computable algorithms using Spiking Neural Networks (SNNs) mimicking neural dynamics with high speed and low power consumption by the dedicated hardware. The analog implementation of neuromorphic computing has been studied in the field of edge computing etc. and is considered to be superior to the digital implementation in terms of power consumption. Furthermore, It is expected to have extremely low power consumption that Processing-In-Memory (PIM) based synaptic operations using non-volatile memory (NVM) devices for both weight memory and multiply-accumulate operations. However, unintended non-linearities and hysteresis occur when attempting to implement analog spiking neuron circuits as simply as possible. As a result, it is thought to cause accuracy loss when inference is performed by mapping the weight parameters of the SNNs which trained offline to the element parameters of the NVM. In this study, we newly designed neuromorphic hardware operating at 100 MHz that employs memcapacitor as a synaptic element, which is expected to have ultra-low power consumption. We also propose a method for training SNNs that incorporate the nonlinearity of the designed circuit into the neuron model and convert the synaptic weights into circuit element parameters. The proposed training method can reduce the degradation of accuracy even for very simple neuron circuits. The proposed circuit and method classify MNIST with ∼33.88 nJ/Inference, excluding the encoder, with ∼97% accuracy. The circuit design and measurement of circuit characteristics were performed in Rohm 180nm process using HSPICE. A spiking neuron model that incorporates circuit non-linearity as an activation function was implemented in PyTorch, a machine learning framework for Python.

Neuromorphic computing [1], inspired by the energy-efficient and parallelized architecture of the human brain, represents a paradigm shift in computing. This approach not only drives innovative hardware designs but also inspires novel network architectures like spiking neural networks (SNNs), which process information as discrete spike events. However, scalability in neuromorphic systems is constrained by the large hardware footprint. Realizing multimodal functionality using the same set of hardware primitives can offer a promising solution to these limitations.In this work, we present a dual encoder-neuron circuit based on a nanoporous graphene (NPG) memristor device [2]. The graphene leaky integrate-and-fire (GLIF) circuit has a versatile multimodal design that combines the roles of a LIF neuron and a spike encoder within the same architecture. This design mimics the dual functionality of biological neurons, which encode and process signals locally. The spike encoding capability of the circuit is further revealed using single and double layer SNN networks for pattern recognition tasks using the Modified National Institute of Standards and Technology (MNIST) dataset.Inset of Figure 1(a) shows the schematic of the fabricated NPG device with a channel length of 100 µm. The device has a lateral structure with NPG channel and gold (Au) electrodes. Figure 1(a) shows the current-voltage (I-V) characteristics of the fabricated NPG device which shows a threshold switching behavior with a wide hysteresis window and a threshold voltage (Vth) of 4.9 V. A behavioral SPICE model is developed based on a voltage-controlled switch model. It is seen from Figure 1(a) that the I-V curves simulated using the SPICE model matches the experimentally observed device switching characteristics. This reveals that the simplified SPICE model can be a good substitute for complex Verilog based models. Based on the SPICE model, we designed a simple neuron circuit with a resistor (Rs, 12 kΩ) in series with a parallel combination of a capacitor (Cm, 10 nF) and the NPG device (SPICE model). Figure 1(b) shows the variation of the membrane potential (Vm) and the output spike current in response to the input pulse voltage (Vin, 6 V). Vm shows a clear leaky integration behavior and full reset to zero voltage after the spike response. These are essential attributes of a bioplausible LIF neuron circuit. In order to develop the multimodal GLIF circuit, the output spike frequency of the LIF neuron circuit is plotted as a function of the Rs values. This mapping allows pixel values from MNIST input images to be directly translated into normalized conductance values (G/G0). Hence, each pixel is represented as a normalized conductance value, establishing a clear correlation between pixel intensity and spike frequency. This core idea is further used to develop the GLIF encoder based on the same neuron circuit as before. Based on the dual encoder-neuron circuit, we designed a single layer SNN for recognising the handwritten digits from the MNIST dataset. Figure 1(c) shows the spike-encoded images for the MNIST digits “2” and “5”. The probability of spiking in each timestep is proportional to the spiking frequency. The spike encoded images at each time step reveal the contours of the original input images using sparse representation. These sparse matrices highlight the energy efficiency and computational advantages of the GLIF system. The performance of the dual encoder-neuron system is evaluated by training the single layer SNN using both the GLIF encoder and neuron circuit as integral components. The single layer SNN recorded a high pattern recognition accuracy of 90.77% which is comparable to the purely software-based implementation (92.69%). To explore the scalability of the GLIF system, we implemented a double-layer SNN using the GLIF system, achieving a high recognition accuracy of 97.37%. The GLIF circuit exemplifies how reconfigurable primitive components can drive the development of scalable neuromorphic hardware.

Application of quantum-dot cellular automata (QCA) technology as an alternative to CMOS technology on the nanoscale has a promising future; QCA is an interesting technology for building memory. The proposed design and simulation of a new memory cell structure based on QCA with a minimum delay, area, and complexity is presented to implement a static random access memory (SRAM). This paper presents the design and simulation of a 16-bit $\times 32$ -bit SRAM with a new structure in QCA. Since QCA is a pipeline, this SRAM has a high operating speed. The 16-bit $\times 32$ -bit SRAM has a new structure with a 32-bit width designed and implemented in QCA. It has the ability of a conventional logic SRAM that can provide read/write operations frequently with minimum delay. The 16-bit $\times 32$ -bit SRAM is generalized and an $n\times 16$ -bit $\times 32$ -bit SRAM is implemented in QCA. Novel 16-bit decoders and multiplexers (MUXs) in QCA are presented that have been designed with a minimum number of majority gates and cells. The new SRAM, decoders, and MUXs are designed, implemented, and simulated in QCA using a signal distribution network to avoid the coplanar problem of crossing wires. The QCA-based SRAM cell was compared with the SRAM cell based on CMOS. Results show that the proposed SRAM is more efficient in terms of area, complexity, clock frequency, latency, throughput, and power consumption.

Though robustness and resilience are commonly quoted as features of neuromorphic computing systems, the expected performance of neuromorphic systems in the face of hardware failures is not clear. In this work, we study the effect of failures on the performance of four different training algo-rithms for spiking neural networks on neuromorphic systems: two back-propagation-based training approaches (Whetstone and SLAYER), a liquid state machine or reservoir computing approach, and an evolutionary optimization-based approach (EONS). We show that these four different approaches have very different resilience characteristics with respect to simulated hardware failures. We then analyze an approach for training more resilient spiking neural networks using the evolutionary optimization approach. We show how this approach produces more resilient networks and discuss how it can be extended to other spiking neural network training approaches as well.

Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks, due to their event-driven spiking computation. However, some foundation SNN backbones (including Spikformer and SEW ResNet) suffer from non-spike computations (integer-float multiplications) caused by the structure of their residual connections. These non-spike computations increase SNNs' power consumption and make them unsuitable for deployment on mainstream neuromorphic hardware. In this paper, we analyze the spike-driven behavior of the residual connection methods in SNNs. We then present Spikingformer, a novel spiking transformer backbone that merges the MS Residual connection with Self-Attention in a biologically plausible way to address the non-spike computation challenge in Spikformer while maintaining global modeling capabilities. We evaluate Spikingformer across 13 datasets spanning large static images, neuromorphic data, and natural language tasks, and demonstrate the effectiveness and universality of Spikingformer, setting a vital benchmark for spiking neural networks. In addition, with the spike-driven features and global modeling capabilities, Spikingformer is expected to become a more efficient general-purpose SNN backbone towards energy-efficient artificial intelligence.

Phase change memory (PCM), one of the most mature emerging non-volatile memories, has gained considerable attention over the years for use as electronic synapses in biologically inspired neuromorphic systems. The resistance drift of PCM devices, nonetheless, has long been identified as one of the biggest challenges toward realizing many areas of applications. Although this drawback has been extensively studied for memory development and many methods were proposed to mitigate the drift effect, its impact, if any, on online learning has not been fully explored yet. In this letter, we investigate the impact of resistance drift and variations in resistance drift parameters during unsupervised online learning. We use the resistance drift characteristics measured from experiments and incorporate them into the spiking neural network (SNN) for MNIST handwritten digits classification. Our results show that resistance drift, considered as a non-ideality for PCM devices, can be exploited to boost accuracy for online learning of handwritten digits in the SNN.

WITHDRAWN: Comparative analysis on low power SRAMs

Static Random Access Memory (SRAM) technology is the latest technology must advance to satisfy the high-speed performance and low power consumption requirements of contemporary devices. The design, testing, and validation of SRAM cells with configurations ranging from six transistors (6T) to ten transistors (10T) are thoroughly examined in this research work. The study's main goal is to pinpoint the crucial elements that affect SRAM cell performance, such as data retention voltages, power consumption, delay factors, stability measures, and noise margins. Existing SRAM designs and their drawbacks are assessed through a thorough literature review, emphasizing the need for innovative SRAM design techniques. The process includes a thorough examination of important variables and the design and testing of several SRAM cell designs using sophisticated simulation tools. To guarantee robustness and reliability, every design from 6T to 10T is painstakingly created, simulated, and verified against a series of predetermined scenarios. The findings show how several SRAM layouts compare in terms of power efficiency, delay, stability, noise margin, and area efficiency. The study's conclusions offer helpful recommendations for improving SRAM designs in the future by weighing the trade-offs between stability, speed, and power consumption. By providing tested design methodologies that improve the performance and efficiency of SRAM cells, this research makes a substantial contribution to the field of semiconductor technology and meets the expanding demands of contemporary electronic gadgets.