A Memcapacitive Spiking Neural Network with Circuit Nonlinearity-aware Training

Spiking neural networks (SNNs) first came to the attention of scientists due to the search for a structure capable of emulating more closely the behavior of the human brain. The biological nervous system has some characteristics that allow it to process a large amount of data very quickly. It is also a fault-tolerant system, with a high level of parallelism. Low power consumption is another feature of the human brain that is desirable for electronic circuits. In this context, several models of artificial spiking neurons were developed, aiming to construct networks able to combine the best characteristics of the human brain. Most of these models, however, lack validation in larger networks. This paper proposes the implementation of an SNN based on a nanoelectronic spiking neuron model developed in previous works. To validate the behavior of an isolated neuron in a network, logic gates (NOT, OR, AND, and XOR) are used as a benchmark. The goal of this paper is to present a feasibility study on the possibility of implementing such nanoelectronic spiking neuron networks based on this spiking neuron model. Nanoelectronics represents an appealing implementation due to the gains regarding occupied area and power consumption, which are inherent characteristics of this technology. The neuron model was modified for simulation at room temperature. An information code based on the amplitude of the pulses presented at the output of the neuron was developed. During deployment of this approach, some limitations regarding the neuron model were detected; some possible solutions are proposed as future work.

Deep learning models are generally trained using a single-precision floating-point number system. However, in the inference process, they use simpler number systems like integers and fixed-points, because of their small design area and low power consumption, despite the accuracy loss and quantization parameter overhead due to their quantization. In general, a floating-point MAC unit makes it unsuitable for inference engines and especially for the area, power, and heat-sensitive devices such as Processor-In-Memory (PIM). In this paper, we propose an efficient MAC design based on the bfloat16 suitable for neural network operations while considering the characteristics of data used for deep learning. Our techniques simplified the design by removing the circuits for handling an underflow, an overflow, and a normalization from the critical path and treating them as exceptions. Also, we improved the computational accuracy by extending the bit-width of the mantissa inside the MAC unit and eliminated unnecessary normalization at every computation. Compared with a MAC unit without our optimization by using the Samsung 65nm library, we reduced the delay of a non-pipelined MAC unit by 47.3%, the area by 9.1 %, and the power consumption by 24.2%, respectively. Furthermore, we show that the proposed bfloat16 MAC outperformed the 16-bit integer MAC in terms of area and power consumption. We also show the design of a 1GHz 3-stage pipelined MAC unit with its performance analysis.

Two-dimensional (2D) materials have received significant attention for their potential use in next-generation electronics, particularly in nonvolatile memory and neuromorphic computing. This is due to their simple metal-insulator-metal (MIM) sandwiched structure, excellent switching performance, high-density capability, and low power consumption. In this work, using comprehensive material simulations and device modeling, the thinnest monolayer hexagonal boron nitride (h-BN) atomristor is studied by using a MIM configuration with Ta electrodes. Our first-principles calculations predicted both a high resistance state (HRS) and a low resistance state (LRS) in this device. We observed that the presence of van der Waals (vdW) gaps between the Ta electrodes and monolayer h-BN with a boron vacancy (VB) contributes to the HRS. The combination of metal electrode contact and the adsorption of Ta atoms onto a single VB defect (TaB) can alter the interface barrier between the electrode and dielectric layer, as well as create band gap states within the band gap of monolayer h-BN. These band gap states can shorten the effective tunneling path for electron transport from the left electrode to the right electrode, resulting in an increase in the current transmission coefficient of the LRS. This resistive switching mechanism in monolayer h-BN atomristors can serve as a theoretical reference for device design and optimization, making them promising for the development of atomristor technology with ultra-high integration density and ultra-low power consumption.

As the third generation of artificial neural networks (ANNs), spiking neural networks (SNNs) have many advantages over the traditional ones. Selecting proper spiking neuron models for the design of SNNs is important. In this paper, schematic and training algorithm of spiking integrated and fired (IAF) neuron model and probability spiking neuron model (pSNM) are introduced. By comparing the classification results for mobile robots' corridor-scene-classifier based on IAF model and pSNM, and the control results of mobile robots' wall-following controller based on spiking IAF model and pSNM, the similarities and differences between the two models are discussed. The similar and different features of the two spiking neuron models are obtained. IAF model is more suitable for the design of mobile robots controller than that of pSNM. While pSNM has better noise robust than IAF model. Spiking IAF model and pSNM are suitable for different situations.

This brief presents an innovative reconfigurable parametric model of a digital Spiking Neuron (SN). The proposed neuron model is based on the classical Leaky Integrate and Fire (LIF) model with some modifications, allowing neuron configuration using seven different leak modes and three activation functions with a dynamic threshold setting. A complementary online learning model based on adjustable Spike Timing Dependent Plasticity (STDP) learning rules has been developed as part of the proposed neuron architecture. Efficient hardware implementation of the proposed SN significantly reduces area and power costs. The proposed SN model consumes less than 10nW and requires only 700 ASIC 2-input NAND gates for implementation using ten neuron-inputs. Simulation results show an average power consumption of about 3.5 mW/cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> . Simulation of the proposed digital SN demonstrates its ability to replicate accurately the behavior of a biological neuron model.

Spiking neural networks (SNNs) show great potential due to their energy efficiency, fast processing capabilities, and robustness. There are two main approaches to constructing SNNs. Direct training methods require much memory, while conversion methods offer a simpler and more efficient option. However, current conversion methods mainly focus on converting convolutional neural networks (CNNs) to SNNs. Converting Transformers to SNN is challenging because of the presence of non-linear modules. In this paper, we propose an Expectation Compensation Module to preserve the accuracy of the conversion. The core idea is to use information from the previous T time-steps to calculate the expected output at time-step T. We also propose a Multi-Threshold Neuron and the corresponding Parallel Parameter normalization to address the challenge of large time steps needed for high accuracy, aiming to reduce network latency and power consumption. Our experimental results demonstrate that our approach achieves state-of-the-art performance. For example, we achieve a top-1 accuracy of 88.60% with only a 1% loss in accuracy using 4 time steps while consuming only 35% of the original power of the Transformer. To our knowledge, this is the first successful Artificial Neural Network (ANN) to SNN conversion for Spiking Transformers that achieves high accuracy, low latency, and low power consumption on complex datasets. The source codes of the proposed method are available at https://github.com/h-z-h-cell/Transformer-to-SNN-ECMT.

Speech trigger and speech command recognition technologies have become pivotal human-machine interfaces for AIoT applications. Typically, a speech-related system consists of a mel-filter-based handcrafted feature extractor and an artificial neuron-based classifier. Recent advancements [1–3] have demonstrated promising results by training the feature extractor jointly with the neural network classifier. Moreover, the emergence of spiking neuron network training algorithms and hardware designs has showcased their competitive edge over traditional artificial neural networks. Consequently, the design and training of learnable audio feature extractors and spiking neural network classifiers are the key focuses of this thesis. The first focus of this thesis is the design of an ultra-low-power, learnable audio feature extractor to support end-to-end training. Conventional low-power approaches, such as extreme pruning and quantization in the audio feature extractor (AuFEx) design, often result in decreased accuracy. To address this, the thesis proposes a hardware-aware, end-to-end training framework to co-optimize the AuFEx design, achieving high performance with minimal hardware cost. Two types of AuFEx designs are explored: a time-domain (TD) filter-based learnable audio feature extractor (AuFEx-TD) and a frequency-domain (FD) filter-based learnable audio feature extractor (AuFEx-FD). Using this framework, the AuFEx-TD achieves high accuracy (97.8%) for one wake word detection, with low power consumption (496 nW) and low latency (375 μs). The optimized AuFEx-FD demonstrates noise-robust 10-keyword spotting accuracy (87.5%@5dB – 92.2%@20dB) with low power consumption (1.26 μW). Both AuFEx-TD and AuFEx-FD designs are implemented using UMC 40nm CMOS technology to validate the framework's effectiveness. For the speech trigger or wake word detection (WWD) task, power and latency are critical design considerations. This thesis presents a sub-μW spiking neural network accelerator with an 8 ms latency. The core WWD engine employs a spiking convolutional neural network (SCNN) model, leveraging sparse activation and addition-only operations within spiking neurons. The proposed SCNN model enhances the existing framewise incremental computation flow [4] by incorporating a spike processing unit (SPU), reducing system power and latency by 16.5% and 43.2%, respectively. Extensive quantization further reduces weight and activation precisions to 4-bit and 1-bit. Additionally, a power-gating mechanism, driven by an energy-based voice activity detection (VAD) module, further minimizes power consumption in random and sparse event (RSE) scenarios. Full chip simulation results show that the chip consumes only 110 nW, with a 2.15% false alarm rate and a 3.00% false reject rate in a 10% voice event stream test. It achieves state-of-the-art recognition accuracy of 99% and 96% for one and two wake word detection tasks, respectively. For speech command recognition or keyword spotting (KWS) tasks, robustness against noise and speaker accents is essential for user experience. This thesis explores both end-to-end training and on-device transfer learning to enhance KWS chip robustness with minimal power overhead. The proposed chip comprises three main blocks: the Learnable Audio Feature Extractor (AuFEx), the Spiking-DSCNN Core (SC), and the Transfer Learning Core (TLC). The AuFEx is an optimized AuFEx-FD, with the on-chip memory for storing FD-filter parameters replaced by an FD-filter parameter generator, reducing power consumption to 1.225 μW. The SC implements a spiking neural network (SNN) model using a Depthwise Separable Convolutional Neural Network (DSCNN) topology. Multi-level encoding in spiking neurons reduces simulation time, while the inherent activation sparsity of these neurons decreases power consumption and latency. The TLC supports both forward and backward propagation for a fully connected layer and softmax layer, enabling the KWS chip to adapt to environmental and user variations using minimal labeled data and weight updates. Fabricated in UMC 40nm CMOS technology, the chip achieves 10-keyword spotting accuracy ranging from 87.5% to 92.2% under background noise levels of 5dB to 20dB. Through on-device transfer learning, the chip can improve KWS accuracy on personalized datasets by 11.73%, using only a few examples per keyword and limited training epochs. The simulated power consumption of the chip in streaming inference mode is 10.26 μW at a 250 kHz clock frequency and 0.85V supply voltage. In conclusion, this thesis presents an end-to-end spiking neural network-based speech system for speech trigger (WWD) and speech command recognition (KWS) tasks. By integrating the AuFEx-TD and the SCNN accelerator, as described in the first and second parts of the thesis, a low-latency, sub-μW wake word detection system has been developed. In the final part, a noise-robust, personalized, few-μW-level speech command recognition ASIC is introduced, demonstrating the feasibility of high-performance, low-power AIoT speech interfaces.

Realization of spiking neural network (SNN) hardware with high energy efficiency and high integration may provide a promising solution to data processing challenges in future internet of things (IoT) and artificial intelligence (AI). Recently, design of multi-core reconfigurable SNN chip based on resistive random-access memory (RRAM) is drawing great attention, owing to the unique properties of RRAM, e.g., high integration density, low power consumption, and processing-in-memory (PIM). Therefore, RRAM-based SNN chip may have further improvements in integration and energy efficiency. The design of such a chip will face the following problems: significant delay in pulse transmission due to complex logic control and inter-core communication; high risk of digital, analog, and RRAM hybrid design; and non-ideal characteristics of analog circuit and RRAM. In order to effectively bridge the gap between device, circuit, algorithm, and architecture, this paper proposes a simulation model—FangTianSim, which covers analog neuron circuit, RRAM model and multi-core architecture and its accuracy is at the clock level. This model can be used to verify the functionalities, delay, and power consumption of SNN chip. This information cannot only be used to verify the rationality of the architecture but also guide the chip design. In order to map different network topologies on the chip, SNN representation format, interpreter, and instruction generator are designed. Finally, the function of FangTianSim is verified on liquid state machine (LSM), fully connected neural network (FCNN), and convolutional neural network (CNN).

Recent advancements in spiking neural networks (SNNs) have highlighted their advantages, including energy efficiency, real-time processing, and compatibility with neuromorphic hardware. These features make SNNs particularly well-suited for human-robot interaction (HRI) in rehabilitative wearable robotics, where real-time adaptability and low power consumption are essential. However, there is still a lack of comprehensive reviews on SNNs’ application to HRI. This paper addresses this gap by providing a detailed overview of the latest advancements in SNNs from the perspective of embodied intelligence in rehabilitative wearable robots. We systematically examine recent progress in SNNs, including spiking neuron models, encoding methods, and learning mechanisms. These advancements are then analyzed with a focus on HRI, addressing specific challenges in rehabilitative wearable robots from three key perspectives: human motion decoding, robotic control, and neuromorphic implementation for embedded systems. By reviewing current research, this paper highlights the potential benefits and limitations of SNNs in achieving embodied intelligence and identifies crucial areas for further investigation, offering new insights and directions for their future applications in rehabilitative wearable robotics.

In recent years, spiking neural networks (SNNs), which originated from the theoretical basis of neuroscience, have attracted neuromorphic computing and brain-like computing due to their advantages, such as neural dynamics and coding mechanism, which are similar to biological neurons. SNNs have become one of the mainstream frameworks in the field of brain-like computing. However, most of the Leaky Integrate-and-Fire (LIF) neuron models currently used by SNNs based on direct training of backpropagation (BP) do not consider the changes in the recurrent connections and the dynamic strength of neuron connections over time. This study presented the LIF neuron model with recurrent connections and a method for dynamically changing the presynaptic currents. Recurrent LIF neurons have an additional cyclic connection compared with classic LIF neurons. Their postsynaptic current stimulates a change in membrane potential at the next time point. Their dynamics were more similar to the activities of biological neurons. We also proposed an efficient and flexible BP training method for recurrent LIF neurons. On the basis of the above methods, we proposed the recurrent SNN with dynamic presynaptic currents based on backpropagation (RDS-BP). We test the proposed RDS-BP on three image data sets (MNIST, Fashion-MNIST and CIFAR-10) and two text data sets (IMDB and TREC). The results showed that the performance of RDS-BP not only exceeded the naive SNN models based on BP but also exceeded the SNN methods proposed in previous studies in recent years, which had excellent performance in previous experiments. Our work provides a new LIF neuron model with a recurrent connection and dynamic presynaptic current and a BP training arrangement for the proposed neuron, which could merit developments with neuromorphic and brain-like computing.

Three-dimensional Integrated Circuits (3D-ICs) is a cutting-edge design methodology of placing the circuitry vertically aiming for a high-speed and energy-efficient system with the smallest design area. In this paper, a novel 3D neuromorphic system is proposed and analyzed, which utilizes the fabricated two-layer memristor as the electronic synapses in a spiking neural network (SNN). The two-layer structure of the memristors leads to a significant improvement in the design area (2×), power consumption (1.48 ×), and latency (2.58×), compared to the traditional one-layer configuration. Meanwhile, the heat dissipation layers are added to our memristors reducing 30% cycle-to-cycle switching variation. Our memristive synapses are utilized for storing the exported weights of the SNNs that have threshold function as the activation function. The proposed neuromorphic system is evaluated using a hardware-software co-design approach importing the weights of SNNs into NeuroSIM. The simulation results demonstrate the significant improvement of memristive synapses on design area, power consumption, and latency, compared with the SRAM (Static Random-access Memory) and other state-of-the-art memristive synapses (10% to 66%).

An adaptive threshold mechanism for accurate and efficient deep spiking convolutional neural networks

Spiking neural networks (SNNs), inspired by the neuronal network in the brain, provide biologically relevant and low-power consuming models for information processing. Existing studies either mimic the learning mechanism of brain neural networks as closely as possible, for example, the temporally local learning rule of spike-timing-dependent plasticity (STDP), or apply the gradient descent rule to optimize a multilayer SNN with fixed structure. However, the learning rule used in the former is local and how the real brain might do the global-scale credit assignment is still not clear, which means that those shallow SNNs are robust but deep SNNs are difficult to be trained globally and could not work so well. For the latter, the nondifferentiable problem caused by the discrete spike trains leads to inaccuracy in gradient computing and difficulties in effective deep SNNs. Hence, a hybrid solution is interesting to combine shallow SNNs with an appropriate machine learning (ML) technique not requiring the gradient computing, which is able to provide both energy-saving and high-performance advantages. In this article, we propose a HybridSNN, a deep and strong SNN composed of multiple simple SNNs, in which data-driven greedy optimization is used to build powerful classifiers, avoiding the derivative problem in gradient descent. During the training process, the output features (spikes) of selected weak classifiers are fed back to the pool for the subsequent weak SNN training and selection. This guarantees HybridSNN not only represents the linear combination of simple SNNs, as what regular AdaBoost algorithm generates, but also contains neuron connection information, thus closely resembling the neural networks of a brain. HybridSNN has the benefits of both low power consumption in weak units and overall data-driven optimizing strength. The network structure in HybridSNN is learned from training samples, which is more flexible and effective compared with existing fixed multilayer SNNs. Moreover, the topological tree of HybridSNN resembles the neural system in the brain, where pyramidal neurons receive thousands of synaptic input signals through their dendrites. Experimental results show that the proposed HybridSNN is highly competitive among the state-of-the-art SNNs.

Real-time target tracking is a usual task for humans despite the neural delays during the nervous system’s axonal transfer and neural processing. A plausible explanation is that the human brain employs predictive mechanisms to compensate for the delay. Inspired by the brain, this paper adopts a prediction network based on spiking neural networks (SNNs) to implement a real-time tracking task on a neuromorphic chip with low power consumption. The SNN-based prediction network outperforms the long short-term memory (LSTM) network on a small dataset and reduces 90% to 98% computations compared with LSTM. The quantized SNN-based network is deployed on a neuromorphic chip, and it takes 25ms and only 442 626nJ for a single prediction. The tracking performance of the system is also verified in real-life scenarios. Furthermore, the proposed real-time target tracking system can be easily ported to other neuromorphic platforms.

Spiking Neural Networks (SNNs) mimic the behavior of biological neurons. Unlike traditional Artificial Neural Networks (ANNs) that operate in a continuous time domain and use activation functions to process information, SNNs operate discrete event-driven, where data is encoded and communicated through spikes or discrete events. This unique approach offers several advantages, such as efficient computation and lower power consumption, making SNNs particularly attractive for energy-constrained and neuromorphic applications. However, training SNNs poses significant challenges due to the discrete nature of spikes and the non-differentiable behavior they exhibit. As a result, converting pre-trained ANNs into SNNs has gained attention as a convenient approach. While this approach simplifies the training process, it introduces certain drawbacks, including high latency. The conversion of ANNs to SNNs typically leads to a loss of accuracy, which can be attributed to various factors, including quantization, clipping, and timing errors. Previous studies have proposed techniques to mitigate quantization and clipping errors during the conversion process. However, they do not consider timing errors, degrading SNN accuracies at low latency conditions. This work introduces the MiCE conversion method, which offers a comprehensive joint optimization strategy to simultaneously alleviate quantization, clipping, and timing errors. At a moderate latency of 8 time-steps, our converted ResNet-20 achieves classification accuracies of 79.02% and 95.74% on the CIFAR-100 and CIFAR-10 datasets, respectively.