Simulation of spiking neural networks — architectures and implementations

To understand the mechanism of information processing by a biological neural network, computer simulation of a large-scale spiking neural network is an important method. However, because of a high computation cost of the simulation of a large-scale spiking neural network, the simulation requires high performance computing implemented by a supercomputer or a computer cluster. Recently, hardware for parallel computing such as a multi-core CPU and a graphics card with a graphics processing unit (GPU) is built in a gaming computer and a workstation. Thus, parallel computing using this hardware is becoming widespread, allowing us to obtain powerful computing power for simulation of a large-scale spiking neural network. However, it is not clear how much increased performance the parallel computing method using a new GPU yields in the simulation of a large-scale spiking neural network. In this study, we compared computation time between the computing methods with CPUs and GPUs in a simulation of neuronal models. We developed computer programs of neuronal simulations for the computing systems that consist of a gaming graphics card with new architecture (the NVIDIA GTX 1080) and an accelerator board using a GPU (the NVIDIA Tesla K20C). Our results show that the computing systems can perform a simulation of a large number of neurons faster than CPU-based systems. Furthermore, we investigated the accuracy of a simulation using single precision floating point. We show that the simulation results of single precision were accurate enough compared with those of double precision, but chaotic neuronal response calculated by a GPU using single precision is prominently different from that calculated by a CPU using double precision. Furthermore, the difference in chaotic dynamics appeared even if we used double precision of a GPU. In conclusion, the GPU-based computing system exhibits a higher computing performance than the CPU-based system, even if the GPU system includes data transfer from a graphics card to host memory.

Substantial evidence indicates that the time structure of neuronal spike trains is relevant in neuronal signal processing. Bio-inspired spiking neural networks are taking these results into account. Applications of these networks to low vision problems, e.g. segmentation, requires that the simulation of large-scale networks must be performed in a reasonable time. On this basis, we investigated the achievable performance of existing hardware platforms for the simulation of spiking neural networks with sizes from 8k neurons up to 512k neurons/50M synapses. We present results for workstations (Sparc-Ultra), digital signal processors (TMS-C8x), neurocomputers (CNAPS, SYNAPSE), small- and large-scale parallel-computers (4xPentium, CM-2, SP2) and discuss the specific implementation issues. According to our investigation, only supercomputers like CM-2 can match the performance requirements for the simulation of very large-scale spiking neural networks. Therefore, there is still. the need for low-cost hardware accelerators.

A workstation has been implemented for development of Brain Machine Interfaces (BMIs) operating with spike-based models of biological neurons. The workstation has been designed to provide real-time interactions between biological and simulated spiking neurons during in vivo BMI experiments. Parallel execution of online spike sorting, real-time spiking neural network (SNN) simulation and experimental environment controlling tasks are supported by the system to fulfill the requirements of closed-loop BMI experiments. The modular software architecture of the system enables easy replacement of SNN simulation and other computation modules required in BMI studies. In this paper, we describe the software structure and computational infrastructure of the workstation, and illustrate the results of a performance test run on the system with an implemented real-time SNN simulator.

The major bottleneck in simulation of large-scale neural networks is the communication problem due to one-to-many neuron connectivity. Network-on-Chip concept has been proposed to address the problem. This work explores the drawback that is introduced by interconnection networks - a delay jitter. The preliminary experiment is held in the spiking neural network simulator introducing variable communicational delay to the simulation. The performance degradation is reported.

Event-driven simulation of neural population synchronization facilitated by electrical coupling

Computation with spiking neurons takes advantage of the abstraction of action potentials into streams of stereotypical events, which encode information through their timing. This approach both reduces power consumption and alleviates communication bottlenecks. A number of such spiking custom mixed-signal address event representation (AER) chips have been developed in recent years.In this paper, we present i) a flexible event-driven platform consisting of the integration of a visual AER sensor and the SpiNNaker system, a programmable massively parallel digital architecture oriented to the simulation of spiking neural networks; ii) the implementation of a neural network for feature-based attentional selection on this platform.KeywordsAttentionSelectionNeuromorphicSpiNNakerAER

This manuscript is focused on scalability of Spiking Neural Network (SNN) for acceleration of its learning time. Simulation of SNN algorithm was implemented on GPUs devices Ge Force 9400M and Ge Force650 GTX in order to compare the learning time. Multiclass database are used for classification and the results are compared.

A major challenge in computational neuroscience is to achieve high performance for real-time simulations of full size brain networks. Recent advances in GPU technology provide massively parallel, low-cost and efficient hardware that is widely available on the computer market. However, the comparatively low-level programming that is necessary to create an efficient GPU-compatible implementation of neuronal network simulations can be challenging, even for otherwise experienced programmers. To resolve this problem a number of tools for simulating spiking neural networks (SNN) on GPUs have been developed [1,2], but using a particular simulator usually comes with restrictions to particular supported neuron models, synapse models or connectivity schemes. Besides being inconvenient, this can unduly influence the path of scientific enquiry. Here we present GeNN (GPU enhance neuronal networks), which builds on NVIDIA's common unified device architecture (CUDA) to enable a more flexible framework. CUDA allows programmers to write C-like code and execute it on NVIDIA’s massively parallel GPUs. However, in order to achieve good performance, it is critical but not trivial to make the right choices on how to parallelize a computational problem, organize its data in memory and optimize the memory access patterns. GeNN is based on the idea that much of this optimization can be cast into heuristics that allow the GeNN meta-compiler to generate optimized GPU code from a basic description of the neuronal network model in a minimal domain specific language of C function calls. For further simplification, this description may also be obtained by translating variables, dynamical equations and parameters from an external simulator into GeNN input files. We are developing this approach for the Brian 2 [3] and SpineCreator/SpineML [4] systems. Using a code generation approach in GeNN has important advantages: 1. A large number of different neuron and synapse models can be provided without performance losses in the final simulation code. 2. The generated simulator code can be optimized for the available GPU hardware and for the specific model. 3. The framework is intrinsically extensible: New GPU optimization strategies, including strategies of other simulators, can be added in the generated code for situations where they are effective. The first release version of GeNN is available at http://sourceforge.net/projects/genn. It has been built and optimized for simulating neuronal networks with an anatomical structure (separate neuron populations with sparse or dense connection patterns with the possibility to use some common learning rules). We have executed performance and scalability tests on an NVIDIA Tesla C2070 GPU with an Intel Xeon(R) E5-2609 CPU running Ubuntu 12.04 LTS. Our results show that as the network size increases, GPU simulations never fail to outperform CPU simulations. But we are also able to demonstrate the performance limits of using GPUs with GeNN under different scenarios of network connectivity, learning rules and simulation parameters, confirming the that GPU acceleration can differ largely depending on the particular details of the model of interest.

Electrical stimulation of neural systems is a key tool for understanding neural dynamics and ultimately for developing clinical treatments. Many applications of electrical stimulation affect large populations of neurons. However, computational models of large networks of spiking neurons are inherently hard to simulate and analyze. We evaluate a reduced mean-field model of excitatory and inhibitory adaptive exponential integrate-and-fire (AdEx) neurons which can be used to efficiently study the effects of electrical stimulation on large neural populations. The rich dynamical properties of this basic cortical model are described in detail and validated using large network simulations. Bifurcation diagrams reflecting the network's state reveal asynchronous up- and down-states, bistable regimes, and oscillatory regions corresponding to fast excitation-inhibition and slow excitation-adaptation feedback loops. The biophysical parameters of the AdEx neuron can be coupled to an electric field with realistic field strengths which then can be propagated up to the population description. We show how on the edge of bifurcation, direct electrical inputs cause network state transitions, such as turning on and off oscillations of the population rate. Oscillatory input can frequency-entrain and phase-lock endogenous oscillations. Relatively weak electric field strengths on the order of 1 V/m are able to produce these effects, indicating that field effects are strongly amplified in the network. The effects of time-varying external stimulation are well-predicted by the mean-field model, further underpinning the utility of low-dimensional neural mass models.

This paper presents a parallel algorithm for the simulation of large complex interconnect networks with frequency-dependent parameters. The equivalent transmission line model, which is suitable for time-domain simulation, is introduced for the modeling of dispersive interconnects. By means of the equivalent multi-port model, the interconnects are separated from other circuit elements, and thus makes the parallel simulation feasible. Based on the proposed technique, a program named FdSPICE is developed for parallel simulation of large interconnect networks. The accuracy and efficiency of FdSPICE are demonstrated by numerical examples.

Clusters of GPUs are becoming commonly used to execute computationally demanding applications. Due to the frequent changes in GPU architecture, many clusters contain heterogeneous types of GPUs, leading to the problem of load distribution among the machines. In this work, we propose a load distribution algorithm for scientific applications executed in heterogeneous GPU clusters. The algorithm finds a distribution of data that minimizes the execution time of the application, by guaranteeing that all GPUs spend the same amount of time processing its assigned kernels and data. We use the algorithm to execute the simulation of large scale neuronal networks. We show that the algorithm effectively balances the load among the GPUs and reduces the execution time of the application.

Describes a scalable parallel architecture for real-time, large scale neural network simulations. Currently the SIMD architecture is largely adopted for implementing digital neurocomputers. However, it is not efficient for simulating large scale neural networks in real-time because of its limited scalability and flexibility. The authors investigate, as a solution, a wavefront array processing (WAP) architecture based on asynchronous communications. The authors compare both architectures in scalability, performance, and flexibility for simulating multi-layer perceptrons. The authors also briefly discuss implementing a high performance digital neurocomputer based on the WAP.

This paper reports further development and applications of a computational model permitting analysis of frequency response characteristics of simulated neurons. The original single‐neuron model, as described by Peddicord and Sampson in 1974, is briefly reviewed here. There follows a thorough description of a new interactive implementation of the model on a PDP‐11/45. The implementation permits high‐speed simulation of large neural networks, and is designed for easy use by investigators with no special knowledge of computers or programming. Experiments with the new system have included the design of improved bandpass filters, pacemaker neurons and networks, and a cerebellar circuit. Simulation results are presented and, in the last case, compared with available physiological data. The paper concludes with a critique of the system, suggestions for further experiments, and a comparison with the simulation system developed by Perkel. Two appendices document the system data structures and show a sample simulation run.

Parallel processing is a promising technique for reducing the execution time of a large and complex wireless network simulation. In this article, a parallel processing technique is presented for time-driven simulations of large and complex wireless networks. The technique explicitly considers the physical-layer details of wireless network simulators such as shadowing and co-channel interference. The technique uses multiple processors interconnected with highspeed links. Principles and issues are elaborated, including workload partitioning, synchronization, and database design. Numerical results from two different wireless network simulators employing the technique are presented to demonstrate the performance of the technique. The results indicate that the technique can significantly improve the run-time performance of time-driven simulations of large and complicated wireless networks of different types.

Spiking neural network simulations incorporating variable transmission delays require synaptic events to be scheduled prior to delivery. Conventional methods have memory requirements that scale with the total number of synapses in a network. We introduce novel scheduling algorithms for both discrete and continuous event delivery, where the memory requirement scales instead with the number of neurons. Superior algorithmic performance is demonstrated using large-scale, benchmarking network simulations.