Encoding, model, and architecture

Building accurate and efficient deep neural network (DNN) models for intelligent sensing systems to process data locally is essential. Spiking neural networks (SNNs) have gained significant popularity in recent years because they are more biological-plausible and energy-efficient than DNNs. However, SNNs usually have lower accuracy than DNNs. In this paper, we propose to use SNNs for image sensing applications. Moreover, we introduce the DNN-SNN knowledge distillation algorithm to reduce the accuracy gap between DNNs and SNNs. Our DNNSNN knowledge distillation improves the accuracy of an SNN by transferring knowledge between a DNN and an SNN. To better transfer the knowledge, our algorithm creates two learning paths from a DNN to an SNN. One path is between the output layer and another path is between the intermediate layer. DNNs use real numbers to propagate information between neurons while SNNs use 1-bit spikes. To empower the communication between DNNs and SNNs, we utilize a decoder to decode spikes into real numbers. Also, our algorithm creates a learning path from an SNN to a DNN. This learning path better adapts the DNN to the SNN by allowing the DNN to learn the knowledge from the SNN. Our SNN models are deployed on Loihi, which is a specialized chip for SNN models. On the MNIST dataset, our SNN models trained by the DNN-SNN knowledge distillation achieve better accuracy than the SNN models on GPU trained by other training algorithms with much lower energy consumption per image.

As it is confronted to inherent neural delays, how does the visual system create a coherent representation of a rapidly changing environment? In this paper, we investigate the role of motion-based prediction in estimating motion trajectories compensating for delayed information sampling. In particular, we investigate how anisotropic diffusion of information may explain the development of anticipatory response as recorded in a neural populations to an approaching stimulus. We validate this using an abstract probabilistic framework and a spiking neural network (SNN) model. Inspired by a mechanism proposed by Nijhawan [1], we first use a Bayesian particle filter framework and introduce a diagonal motion-based prediction model which extrapolates the estimated response to a delayed stimulus in the direction of the trajectory. In the SNN implementation, we have used this pattern of anisotropic, recurrent connections between excitatory cells as mechanism for motion-extrapolation. Consistent with recent experimental data collected in extracellular recordings of macaque primary visual cortex [2], we have simulated different trajectory lengths and have explored how anticipatory responses may be dependent on the information accumulated along the trajectory. We show that both our probabilistic framework and the SNN model can replicate the experimental data qualitatively. Most importantly, we highlight requirements for the development of a trajectory-dependent anticipatory response, and in particular the anisotropic nature of the connectivity pattern which leads to the motion extrapolation mechanism.

The Spiking Neural Network (SNN) model is a third-generation neural network model that uses spike pulse trains to detect and classify electrical problems in the electrical equipment under consideration. To diagnose the thermal issue in the early stages, it is required to evaluate and monitor the electrical components. The detection method in Infrared Thermography (IRT) is hybrid optimization, which is a Dragonfly Algorithm (DA)–Ant Colony Optimization (ACO) technique that produces a higher exploration and exploitation rate while moving toward an optimal solution with a higher convergence rate. The innovative hybrid DA–ACO method used in this study aims to find the best weights for the SNN model while also extracting the most useful characteristics for defect detection in electrical equipment. To improve classification accuracy, the optimum features picked using the created hybrid DA–ACO are provided to the SNN model. Based on conventional and other optimization methods, the proposed method provided superior results from the execution results.

Spiking Neural Networks (SNNs) have the potential for achieving low energy consumption due to their biologically sparse computation. Several studies have shown that the off-chip memory (DRAM) accesses are the most energy-consuming operations in SNN processing. However, state-of-the-art in SNN systems do not optimize the DRAM energy-per-access, thereby hindering achieving high energy-efficiency. To substantially minimize the DRAM energy-per-access, a key knob is to reduce the DRAM supply voltage but this may lead to DRAM errors (i.e., the so-called approximate DRAM). Towards this, we propose SparkXD, a novel framework that provides a comprehensive conjoint solution for resilient and energy-efficient SNN inference using low-power DRAMs subjected to voltage-induced errors. The key mechanisms of SparkXD are: (1) improving the SNN error tolerance through fault-aware training that considers bit errors from approximate DRAM, (2) analyzing the error tolerance of the improved SNN model to find the maximum tolerable bit error rate (BER) that meets the targeted accuracy constraint, and (3) energy-efficient DRAM data mapping for the resilient SNN model that maps the weights in the appropriate DRAM location to minimize the DRAM access energy. Through these mechanisms, SparkXD mitigates the negative impact of DRAM (approximation) errors, and provides the required accuracy. The experimental results show that, for a target accuracy within 1% of the baseline design (i.e., SNN without DRAM errors), SparkXD reduces the DRAM energy by ca. 40% on average across different network sizes.

Spiking Neural Networks (SNNs) bear the potential of efficient unsupervised\nand continual learning capabilities because of their biological plausibility,\nbut their complexity still poses a serious research challenge to enable their\nenergy-efficient design for resource-constrained scenarios (like embedded\nsystems, IoT-Edge, etc.). We propose SpikeDyn, a comprehensive framework for\nenergy-efficient SNNs with continual and unsupervised learning capabilities in\ndynamic environments, for both the training and inference phases. It is\nachieved through the following multiple diverse mechanisms: 1) reduction of\nneuronal operations, by replacing the inhibitory neurons with direct lateral\ninhibitions; 2) a memory- and energy-constrained SNN model search algorithm\nthat employs analytical models to estimate the memory footprint and energy\nconsumption of different candidate SNN models and selects a Pareto-optimal SNN\nmodel; and 3) a lightweight continual and unsupervised learning algorithm that\nemploys adaptive learning rates, adaptive membrane threshold potential, weight\ndecay, and reduction of spurious updates. Our experimental results show that,\nfor a network with 400 excitatory neurons, our SpikeDyn reduces the energy\nconsumption on average by 51% for training and by 37% for inference, as\ncompared to the state-of-the-art. Due to the improved learning algorithm,\nSpikeDyn provides on avg. 21% accuracy improvement over the state-of-the-art,\nfor classifying the most recently learned task, and by 8% on average for the\npreviously learned tasks.\n

Spiking neural networks (SNNs) capture some of the efficiency of biological brains for inference and learning via the dynamic, online, and event-driven processing of binary time series. Most existing learning algorithms for SNNs are based on deterministic neuronal models, such as leaky integrate-and-fire, and rely on heuristic approximations of backpropagation through time that enforces constraints such as locality. In contrast, probabilistic SNN models can be trained directly via principled online, local, and update rules that have proven to be particularly effective for resource-constrained systems. This article investigates another advantage of probabilistic SNNs, namely, their capacity to generate independent outputs when queried over the same input. It is shown that the multiple generated output samples can be used during inference to robustify decisions and to quantify uncertainty-a feature that deterministic SNN models cannot provide. Furthermore, they can be leveraged for training in order to obtain more accurate statistical estimates of the log-loss training criterion and its gradient. Specifically, this article introduces an online learning rule based on generalized expectation-maximization (GEM) that follows a three-factor form with global learning signals and is referred to as GEM-SNN. Experimental results on structured output memorization and classification on a standard neuromorphic dataset demonstrate significant improvements in terms of log-likelihood, accuracy, and calibration when increasing the number of samples used for inference and training.

Repeaters equipped with on-board digital baseband processing in a time division duplex (TDD) demand short processing time in order to alleviate inter-symbol interference resulting from having a time delay that is greater than the guard time. To accomplish this, the total system delay of the repeater should be minimized as much as possible without distorting signal quality. Conventionally, the finite impulse response (FIR) type of filter is deployed as a channelization filter, but due to the necessity of large numbers of coefficients to fulfill a prerequisite filter response with a sharp transition band characteristic, an unwanted excessive time delay intrinsically occurs. To make the processing delay as low as possible, this paper proposes a method employing a minimum-phase characterized infinite impulse response (IIR) filter whose magnitude response is almost identical to that of the original FIR filter. Furthermore, in order to linearize the phase response of the designed IIR filter, this paper also introduces an all-pass filter cascaded with the IIR filter for digital down-conversion as well as up-conversion. To achieve further simplicity, this paper introduces polyphase-style IIR filters transformed from conventional single IIR filters that have their own all-pass filters in order to linearize the phase response. The computer simulation results verify that the proposed integrated IIR filter exhibits a relatively short processing delay with a minor deterioration in signal quality-like error vector magnitude (EVM) performance.

Bio-inspired Active Learning method in spiking neural network

Machine learning applications that are implemented with spike-based computation model, e.g., Spiking Neural Network (SNN), have a great potential to lower the energy consumption when they are executed on a neuromorphic hardware. However, compiling and mapping an SNN to the hardware is challenging, especially when compute and storage resources of the hardware (viz. crossbar) need to be shared among the neurons and synapses of the SNN. We propose an approach to analyze and compile SNNs on a resource-constrained neuromorphic hardware, providing guarantee on key performance metrics such as execution time and throughput. Our approach makes the following three key contributions. First, we propose a greedy technique to partition an SNN into clusters of neurons and synapses such that each cluster can fit on to the resources of a crossbar. Second, we exploit the rich semantics and expressiveness of Synchronous Dataflow Graphs (SDFGs) to represent a clustered SNN and analyze its performance using Max-Plus Algebra, considering the available compute and storage capacities, buffer sizes, and communication bandwidth. Third, we propose a self-timed execution-based fast technique to compile and admit SNN-based applications to a neuromorphic hardware at run-time, adapting dynamically to the available resources on the hardware. We evaluate our approach with standard SNN-based applications and demonstrate a significant performance improvement compared to current practices.

As the desire for biologically realistic spiking neural networks (SNNs) increases, tuning the enormous number of open parameters in these models becomes a difficult challenge. SNNs have been used to successfully model complex neural circuits that explore various neural phenomena such as neural plasticity, vision systems, auditory systems, neural oscillations, and many other important topics of neural function. Additionally, SNNs are particularly well-adapted to run on neuromorphic hardware that will support biological brain-scale architectures. Although the inclusion of realistic plasticity equations, neural dynamics, and recurrent topologies has increased the descriptive power of SNNs, it has also made the task of tuning these biologically realistic SNNs difficult. To meet this challenge, we present an automated parameter tuning framework capable of tuning SNNs quickly and efficiently using evolutionary algorithms (EA) and inexpensive, readily accessible graphics processing units (GPUs). A sample SNN with 4104 neurons was tuned to give V1 simple cell-like tuning curve responses and produce self-organizing receptive fields (SORFs) when presented with a random sequence of counterphase sinusoidal grating stimuli. A performance analysis comparing the GPU-accelerated implementation to a single-threaded central processing unit (CPU) implementation was carried out and showed a speedup of 65× of the GPU implementation over the CPU implementation, or 0.35 h per generation for GPU vs. 23.5 h per generation for CPU. Additionally, the parameter value solutions found in the tuned SNN were studied and found to be stable and repeatable. The automated parameter tuning framework presented here will be of use to both the computational neuroscience and neuromorphic engineering communities, making the process of constructing and tuning large-scale SNNs much quicker and easier.

How to effectively and efficiently deal with spatio-temporal event streams, where the events are generally sparse and non-uniform and have the μs temporal resolution, is of great value and has various real-life applications. Spiking neural network (SNN), as one of the brain-inspired event-triggered computing models, has the potential to extract effective spatio-temporal features from the event streams. However, when aggregating individual events into frames with a new higher temporal resolution, existing SNN models do not attach importance to that the serial frames have different signal-to-noise ratios since event streams are sparse and non-uniform. This situation interferes with the performance of existing SNNs. In this work, we propose a temporal-wise attention SNN (TA-SNN) model to learn frame-based representation for processing event streams. Concretely, we extend the attention concept to temporal-wise input to judge the significance of frames for the final decision at the training stage, and discard the irrelevant frames at the inference stage. We demonstrate that TA-SNN models improve the accuracy of event streams classification tasks. We also study the impact of multiple-scale temporal resolutions for frame-based representation. Our approach is tested on three different classification tasks: gesture recognition, image classification, and spoken digit recognition. We report the state-of-the-art results on these tasks, and get the essential improvement of accuracy (almost 19%) for gesture recognition with only 60 ms.

Recently, General Purpose Graphical Processing Units (GP-GPUs) have been identified as an intriguing technology to accelerate numerous data-parallel algorithms. Several GPU architectures and programming models are beginning to emerge and establish their niche in the High-Performance Computing (HPC) community. New massively parallel architectures such as the Nvidia’s Fermi and AMD/ATi’s Radeon pack tremendous computing power in their large number of multiprocessors. Their performance is unleashed using one of the two GP-GPU programming models: Compute Unified Device Architecture (CUDA) and Open Computing Language (OpenCL). Both of them offer constructs and features that have direct bearing on the application runtime performance. In this paper, we compare the two GP-GPU architectures and the two programming models using a two-level character recognition network. The two-level network is developed using four different Spiking Neural Network (SNN) models, each with different ratios of computation-to-communication requirements. To compare the architectures, we have chosen the two extremes of the SNN models for implementation of the aforementioned two-level network. An architectural performance comparison of the SNN application running on Nvidia’s Fermi and AMD/ATi’s Radeon is done using the OpenCL programming model exhausting all of the optimization strategies plausible for the two architectures. To compare the programming models, we implement the two-level network on Nvidia’s Tesla C2050 based on the Fermi architecture. We present a hierarchy of implementations, where we successively add optimization techniques associated with the two programming models. We then compare the two programming models at these different levels of implementation and also present the effect of the network size (problem size) on the performance. We report significant application speed-up, as high as 1095× for the most computation intensive SNN neuron model, against a serial implementation on the Intel Core 2 Quad host. A comprehensive study presented in this paper establishes connections between programming models, architectures and applications.

Vision based-target tracking ability is crucial to bio-inspired snake robots for exploring unknown environments. However, it is difficult for the traditional vision modules of snake robots to overcome the image blur resulting from periodic swings. A promising approach is to use a neuromorphic vision sensor (NVS), which mimics the biological retina to detect a target at a higher temporal frequency and in a wider dynamic range. In this study, an NVS and a spiking neural network (SNN) were performed on a snake robot for the first time to achieve pipe-like object tracking. An SNN based on Hough Transform was designed to detect a target with an asynchronous event stream fed by the NVS. Combining the state of snake motion analyzed by the joint position sensors, a tracking framework was proposed. The experimental results obtained from the simulator demonstrated the validity of our framework and the autonomous locomotion ability of our snake robot. Comparing the performances of the SNN model on CPUs and on GPUs, respectively, the SNN model showed the best performance on a GPU under a simplified and synchronous update rule while it possessed higher precision on a CPU in an asynchronous way.

This paper presents a Spiking Neural Network (SNN) processor specifically designed to overcome the limitations of existing parallel architectures in maintaining high energy efficiency and model adaptability in a compact area footprint for Artificial Intelligence of Things (AIoT). This is achieved through two key design features: a Temporal-Spatial Post-Neuron Processing (PoNP) scheme that efficiently reuses membrane potential, maximizes parallelism, and reduces memory bank requirements; and a Model-Adaptive Crossbar design with pre-configured parameters and a dynamic switching mechanism enables processing of various SNN models through operation orchestration without efficiency degradation. Using an 8-way parallel pipeline design, the processor achieves a throughput of 128 Synaptic Operations (SOPs) per cycle, resulting in a 2.8$\boldsymbol{\times}$ enhancement in energy efficiency. Fabricated in a 40-nm CMOS process, the chip occupies a compact core area of 0.66 mm${}^{2}$. It achieves a power consumption of 6.26 mW, an energy efficiency of 0.49 pJ/SOP, and a throughput of 12.8 GSOPS at 0.75 V, 100 MHz. The chip is evaluated using typical spatial, temporal, and temporal-spatial datasets, including MIT-BIH, MNIST, N-MNIST, NavGesture, and SHD. These results demonstrate that our chip achieves best-in-class in terms of energy efficiency and latency compared to state-of-the-art architectures.

Human information processing depends mainly on billions of neurons which constitute a complex neural network, and the information is transmitted in the form of neural spikes. In this paper, we propose a spiking neural network (SNN), named MD-SNN, with three key features: (1) using receptive field to encode spike trains from images; (2) randomly selecting partial spikes as inputs for each neuron to approach the absolute refractory period of the neuron; (3) using groups of neurons to make decisions. We test MD-SNN on the MNIST data set of handwritten digits, and results demonstrate that: (1) Different sizes of receptive fields influence classification results significantly. (2) Considering the neuronal refractory period in the SNN model, increasing the number of neurons in the learning layer could greatly reduce the training time, effectively reduce the probability of over-fitting, and improve the accuracy by 8.77%. (3) Compared with other SNN methods, MD-SNN achieves a better classification; compared with the convolution neural network, MD-SNN maintains flip and rotation invariance (the accuracy can remain at 90.44% on the test set), and it is more suitable for small sample learning (the accuracy can reach 80.15% for 1000 training samples, which is 7.8 times that of CNN).