Simulation of spiking neural networks on different hardware platforms

To understand the mechanism of information processing by a biological neural network, computer simulation of a large-scale spiking neural network is an important method. However, because of a high computation cost of the simulation of a large-scale spiking neural network, the simulation requires high performance computing implemented by a supercomputer or a computer cluster. Recently, hardware for parallel computing such as a multi-core CPU and a graphics card with a graphics processing unit (GPU) is built in a gaming computer and a workstation. Thus, parallel computing using this hardware is becoming widespread, allowing us to obtain powerful computing power for simulation of a large-scale spiking neural network. However, it is not clear how much increased performance the parallel computing method using a new GPU yields in the simulation of a large-scale spiking neural network. In this study, we compared computation time between the computing methods with CPUs and GPUs in a simulation of neuronal models. We developed computer programs of neuronal simulations for the computing systems that consist of a gaming graphics card with new architecture (the NVIDIA GTX 1080) and an accelerator board using a GPU (the NVIDIA Tesla K20C). Our results show that the computing systems can perform a simulation of a large number of neurons faster than CPU-based systems. Furthermore, we investigated the accuracy of a simulation using single precision floating point. We show that the simulation results of single precision were accurate enough compared with those of double precision, but chaotic neuronal response calculated by a GPU using single precision is prominently different from that calculated by a CPU using double precision. Furthermore, the difference in chaotic dynamics appeared even if we used double precision of a GPU. In conclusion, the GPU-based computing system exhibits a higher computing performance than the CPU-based system, even if the GPU system includes data transfer from a graphics card to host memory.

The major bottleneck in simulation of large-scale neural networks is the communication problem due to one-to-many neuron connectivity. Network-on-Chip concept has been proposed to address the problem. This work explores the drawback that is introduced by interconnection networks - a delay jitter. The preliminary experiment is held in the spiking neural network simulator introducing variable communicational delay to the simulation. The performance degradation is reported.

A workstation has been implemented for development of Brain Machine Interfaces (BMIs) operating with spike-based models of biological neurons. The workstation has been designed to provide real-time interactions between biological and simulated spiking neurons during in vivo BMI experiments. Parallel execution of online spike sorting, real-time spiking neural network (SNN) simulation and experimental environment controlling tasks are supported by the system to fulfill the requirements of closed-loop BMI experiments. The modular software architecture of the system enables easy replacement of SNN simulation and other computation modules required in BMI studies. In this paper, we describe the software structure and computational infrastructure of the workstation, and illustrate the results of a performance test run on the system with an implemented real-time SNN simulator.

Conventional hardware platforms are far from reaching real-time simulation requirements of complex spiking neural networks (SNN). Therefore we designed an accelerator board with a neuro-processor-chip, called NeuroPipe-Chip. In this paper, we introduce two new concepts on chip-level to speed up the simulation of SNN. The concepts are implemented in the architecture of the NeuroPipe-Chip. We present the hardware structure of the NeuroPipe-Chip, which is modelled on register-transfer-level (RTL) using the hardware description language VHDL. We evaluate the performance of the NeuroPipe-Chip in a system simulation, where the rest of the accelerator board is modelled in behavioral VHDL. For a simple SNN for image segmentation, the NeuroPipe-Chip operating at 100 MHz shows an improvement of more than two orders of magnitude compared to an Alpha 500 MHz workstation and approaches real-time requirements for SNN in the order of 10/sup 6/ neurons. Hence, such an accelerator would allow real-time simulations of complex SNN for image processing. Currently, the implementation of the NeuroPipe-Chip in a 0.35 m digital CMOS technology is investigated.

This manuscript is focused on scalability of Spiking Neural Network (SNN) for acceleration of its learning time. Simulation of SNN algorithm was implemented on GPUs devices Ge Force 9400M and Ge Force650 GTX in order to compare the learning time. Multiclass database are used for classification and the results are compared.

Computation with spiking neurons takes advantage of the abstraction of action potentials into streams of stereotypical events, which encode information through their timing. This approach both reduces power consumption and alleviates communication bottlenecks. A number of such spiking custom mixed-signal address event representation (AER) chips have been developed in recent years.In this paper, we present i) a flexible event-driven platform consisting of the integration of a visual AER sensor and the SpiNNaker system, a programmable massively parallel digital architecture oriented to the simulation of spiking neural networks; ii) the implementation of a neural network for feature-based attentional selection on this platform.KeywordsAttentionSelectionNeuromorphicSpiNNakerAER

Neural coupled oscillators are a useful building block in numerous models and applications. They were analyzed extensively in theoretical studies and more recently in biologically realistic simulations of spiking neural networks. The advent of mixed-signal analog/digital neuromorphic electronic circuits provides new means for implementing neural coupled oscillators on compact, low-power, spiking neural network hardware platforms. However, their implementation on this noisy, low-precision and inhomogeneous computing substrate raises new challenges with regards to stability and controllability. In this work, we present a robust, spiking neural network model of neural coupled oscillators and validate it with an implementation on a mixed-signal neuromorphic processor. We demonstrate its robustness showing how to reliably control and modulate the oscillator's frequency and phase shift, despite the variability of the silicon synapse and neuron properties. We show how this ultra-low power neural processing system can be used to build an adaptive cardiac pacemaker modulating the heart rate with respect to the respiration phases and compare it with surface ECG and respiratory signal recordings from dogs at rest. The implementation of our model in neuromorphic electronic hardware shows its robustness on a highly variable substrate and extends the toolbox for applications requiring rhythmic outputs such as pacemakers.

Simulation of spiking neural networks — architectures and implementations

We present an approach to constructing a neuromorphic device that responds to language input by producing neuron spikes in proportion to the strength of the appropriate positive or negative emotional response. Specifically, we perform a fine-grained sentiment analysis task with implementations on two different systems: one using conventional spiking neural network (SNN) simulators and the other one using IBM's Neurosynaptic System TrueNorth. Input words are projected into a high-dimensional semantic space and processed through a fully-connected neural network (FCNN) containing rectified linear units (ReLU) trained via backpropagation. After training, this FCNN is converted to a SNN by substituting the ReLUs with integrate-and-fire neurons. We show that there is practically no performance loss due to conversion to a spiking network on a sentiment analysis test set, i.e. correlations with human annotations differ by less than 0.02 between the original DNN and its spiking equivalent. Additionally, we show that the SNN generated with this technique can be mapped to existing neuromorphic hardware - in our case, the TrueNorth chip. Mapping to the chip involves 4-bit synaptic weight discretization and adjustment of the neuron thresholds. The resulting end-to-end system can take a user input, i.e. a word in a vocabulary of over 300,000 words, and estimate its sentiment on TrueNorth with a power consumption of approximately 50 µW.

Event-driven simulation of neural population synchronization facilitated by electrical coupling

Spiking Neural Network (SNN) has been known as an effective tool for advancing comprehension of the brain. Recent efforts aim to simulate human-scale SNNs, encompassing 86 billion neurons and hundreds of trillion synapses. However, network communication emerges as a bottleneck in SNN simulation. In this paper, we propose DFMG, a Delay-Flush Multi-Group algorithm which utilizes the synaptic delay in multi-area models to alleviate the communication bottleneck. Firstly, DFMG adopts a delay-flush communication pattern to reduce the network packet counts. Then, DFMG utilizes several smaller communication groups instead of a large one to improve communication locality. DFMG also includes a hierarchical clustering algorithm and a greedy patching algorithm which partition the groups. We evaluate the performance with a model of 116 areas using 232 compute nodes on the Tianhe-Xingyi Supercomputing System. The results show that our DFMG improves communication performance and achieves up to 26% speedup.

Recent work in neuromorphic computing has proposed a range of new architectures for Spiking Neural Network (SNN)-based systems. However, neuromorphic design lacks a framework to facilitate exploration of different SNN-based architectures and aid with early design decisions. While there are various SNN simulators, none can be used to rapidly estimate latency and energy of different spiking architectures. We show that while current spiking designs differ in implementation, they have common features which can be represented as a generic architecture template. We describe an initial version of a framework that simulates a range of neuromorphic architectures at an abstract time-step granularity. We demonstrate our simulator by modeling Intel's Loihi platform, estimating time-varying energy and latency with less than 10% mean error for various sizes of a two-layer SNN.

Parallel and distributed simulation tools are emerging that offer the ability to perform detailed, packet-level simulations of large-scale computer networks on an unprecedented scale. The state-of-the-art in large-scale network simulation is characterized quantitatively. For this purpose, a metric based on the number of packet transmissions that can be processed by a simulator per second of wallclock time (PTS) is used as a means to quantitatively assess packet-level network simulator performance. An approach to realizing scalable network simulations that leverages existing sequential simulation models and software is described. Results from a recent performance study are presented concerning large-scale network simulation on a variety of platforms ranging from workstations to cluster computers to supercomputers. These experiments include runs utilizing as many as 1536 processors yielding performance as high as 106 million PTS. The performance of packet-level simulations of web and ftp traffic, and denial of service attacks on networks containing millions of network nodes are briefly described, including a run demonstrating the ability to simulate a million web traffic flows in near real-time. New opportunities and research challenges to fully exploit this capability are discussed.

Spiking neural network simulations incorporating variable transmission delays require synaptic events to be scheduled prior to delivery. Conventional methods have memory requirements that scale with the total number of synapses in a network. We introduce novel scheduling algorithms for both discrete and continuous event delivery, where the memory requirement scales instead with the number of neurons. Superior algorithmic performance is demonstrated using large-scale, benchmarking network simulations.

Introduction It has been shown that a form of temporal difference (TD) learning for predicting the value of the membrane potential of a neuron, at a fixed delay after the neuron received a presynaptic spike, results in a learning rule that is very similar to hebbian spike-timing-dependent plasticity (STDP) [1]. Since this result was obtained using a relatively complex neural model (two-compartmental, with Hodgkin-Huxley-like currents) and a simple setup (a single presynaptic spike followed by a single current pulse), we investigated whether it holds for simpler neural models and for more general situations. This is relevant both for the theoretical understanding of this phenomenon and for verifying that it holds in common simulations of spiking neural networks.