Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems

To understand the mechanism of information processing by a biological neural network, computer simulation of a large-scale spiking neural network is an important method. However, because of a high computation cost of the simulation of a large-scale spiking neural network, the simulation requires high performance computing implemented by a supercomputer or a computer cluster. Recently, hardware for parallel computing such as a multi-core CPU and a graphics card with a graphics processing unit (GPU) is built in a gaming computer and a workstation. Thus, parallel computing using this hardware is becoming widespread, allowing us to obtain powerful computing power for simulation of a large-scale spiking neural network. However, it is not clear how much increased performance the parallel computing method using a new GPU yields in the simulation of a large-scale spiking neural network. In this study, we compared computation time between the computing methods with CPUs and GPUs in a simulation of neuronal models. We developed computer programs of neuronal simulations for the computing systems that consist of a gaming graphics card with new architecture (the NVIDIA GTX 1080) and an accelerator board using a GPU (the NVIDIA Tesla K20C). Our results show that the computing systems can perform a simulation of a large number of neurons faster than CPU-based systems. Furthermore, we investigated the accuracy of a simulation using single precision floating point. We show that the simulation results of single precision were accurate enough compared with those of double precision, but chaotic neuronal response calculated by a GPU using single precision is prominently different from that calculated by a CPU using double precision. Furthermore, the difference in chaotic dynamics appeared even if we used double precision of a GPU. In conclusion, the GPU-based computing system exhibits a higher computing performance than the CPU-based system, even if the GPU system includes data transfer from a graphics card to host memory.

Neural coupled oscillators are a useful building block in numerous models and applications. They were analyzed extensively in theoretical studies and more recently in biologically realistic simulations of spiking neural networks. The advent of mixed-signal analog/digital neuromorphic electronic circuits provides new means for implementing neural coupled oscillators on compact, low-power, spiking neural network hardware platforms. However, their implementation on this noisy, low-precision and inhomogeneous computing substrate raises new challenges with regards to stability and controllability. In this work, we present a robust, spiking neural network model of neural coupled oscillators and validate it with an implementation on a mixed-signal neuromorphic processor. We demonstrate its robustness showing how to reliably control and modulate the oscillator's frequency and phase shift, despite the variability of the silicon synapse and neuron properties. We show how this ultra-low power neural processing system can be used to build an adaptive cardiac pacemaker modulating the heart rate with respect to the respiration phases and compare it with surface ECG and respiratory signal recordings from dogs at rest. The implementation of our model in neuromorphic electronic hardware shows its robustness on a highly variable substrate and extends the toolbox for applications requiring rhythmic outputs such as pacemakers.

A workstation has been implemented for development of Brain Machine Interfaces (BMIs) operating with spike-based models of biological neurons. The workstation has been designed to provide real-time interactions between biological and simulated spiking neurons during in vivo BMI experiments. Parallel execution of online spike sorting, real-time spiking neural network (SNN) simulation and experimental environment controlling tasks are supported by the system to fulfill the requirements of closed-loop BMI experiments. The modular software architecture of the system enables easy replacement of SNN simulation and other computation modules required in BMI studies. In this paper, we describe the software structure and computational infrastructure of the workstation, and illustrate the results of a performance test run on the system with an implemented real-time SNN simulator.

Substantial evidence indicates that the time structure of neuronal spike trains is relevant in neuronal signal processing. Bio-inspired spiking neural networks are taking these results into account. Applications of these networks to low vision problems, e.g. segmentation, requires that the simulation of large-scale networks must be performed in a reasonable time. On this basis, we investigated the achievable performance of existing hardware platforms for the simulation of spiking neural networks with sizes from 8k neurons up to 512k neurons/50M synapses. We present results for workstations (Sparc-Ultra), digital signal processors (TMS-C8x), neurocomputers (CNAPS, SYNAPSE), small- and large-scale parallel-computers (4xPentium, CM-2, SP2) and discuss the specific implementation issues. According to our investigation, only supercomputers like CM-2 can match the performance requirements for the simulation of very large-scale spiking neural networks. Therefore, there is still. the need for low-cost hardware accelerators.

Conventional hardware platforms are far from reaching real-time simulation requirements of complex spiking neural networks (SNN). Therefore we designed an accelerator board with a neuro-processor-chip, called NeuroPipe-Chip. In this paper, we introduce two new concepts on chip-level to speed up the simulation of SNN. The concepts are implemented in the architecture of the NeuroPipe-Chip. We present the hardware structure of the NeuroPipe-Chip, which is modelled on register-transfer-level (RTL) using the hardware description language VHDL. We evaluate the performance of the NeuroPipe-Chip in a system simulation, where the rest of the accelerator board is modelled in behavioral VHDL. For a simple SNN for image segmentation, the NeuroPipe-Chip operating at 100 MHz shows an improvement of more than two orders of magnitude compared to an Alpha 500 MHz workstation and approaches real-time requirements for SNN in the order of 10/sup 6/ neurons. Hence, such an accelerator would allow real-time simulations of complex SNN for image processing. Currently, the implementation of the NeuroPipe-Chip in a 0.35 m digital CMOS technology is investigated.

The major bottleneck in simulation of large-scale neural networks is the communication problem due to one-to-many neuron connectivity. Network-on-Chip concept has been proposed to address the problem. This work explores the drawback that is introduced by interconnection networks - a delay jitter. The preliminary experiment is held in the spiking neural network simulator introducing variable communicational delay to the simulation. The performance degradation is reported.

Introduction It has been shown that a form of temporal difference (TD) learning for predicting the value of the membrane potential of a neuron, at a fixed delay after the neuron received a presynaptic spike, results in a learning rule that is very similar to hebbian spike-timing-dependent plasticity (STDP) [1]. Since this result was obtained using a relatively complex neural model (two-compartmental, with Hodgkin-Huxley-like currents) and a simple setup (a single presynaptic spike followed by a single current pulse), we investigated whether it holds for simpler neural models and for more general situations. This is relevant both for the theoretical understanding of this phenomenon and for verifying that it holds in common simulations of spiking neural networks.

In this paper, we present a methodology for effi-ciently mapping neural networks over a neuromorphic computing architecture. The target architecture is a globally asynchronous locally synchronous (GALS) multi-core designed for simulating spiking neural networks (SNN) in real-time, that is spike timings should be the same as in the human brain. The SNN is implemented as a set of concurrent tasks modelling the behaviour of biological neurons, which are executed on the processing cores and communicate through spikes travelling on a network-on-chip. The problem of neuron-to-core mapping is relevant as a non-efficient allocation may impact real-time and reliability of the neural network execution. We designed a task placement pipeline capable of analysing the network of neurons and producing a placement configuration that enables a reduction of communication between computational nodes. The neuron-to-core mapping problem has been formalised as a problem of minimisation of synaptic elongation. Intuitively, this metric represents the cumulative distance that spikes generated by neurons running on a specific core have to travel to reach their destination core. The proposed placement methodology allows using different techniques to solve the problem. In this work Spectral Analysis, Multilevel Static Mapping, and Simulated Annealing were compared evaluating the overall post-placement synaptic elongation. Results point out that mapping solutions taking into account the directionality of the SNN provide a better placement and quantify this impact. Between all techniques considered only the Simulated Annealing was able to overcome an improvement of 25% compared to a random placement.

We study the influence of network topology on retrieval properties of recurrent neural networks, using replica techniques for diluted systems. The theory is presented for a network with an arbitrary degree distribution $p(k)$ and applied to power law distributions $p(k) \sim k^{-\gamma}$, i.e. to neural networks on scale-free graphs. A bifurcation analysis identifies phase boundaries between the paramagnetic phase and either a retrieval phase or a spin glass phase. Using a population dynamics algorithm, the retrieval overlap and spin glass order parameters may be calculated throughout the phase diagram. It is shown that there is an enhancement of the retrieval properties compared with a Poissonian random graph. We compare our findings with simulations.

Computation with spiking neurons takes advantage of the abstraction of action potentials into streams of stereotypical events, which encode information through their timing. This approach both reduces power consumption and alleviates communication bottlenecks. A number of such spiking custom mixed-signal address event representation (AER) chips have been developed in recent years.In this paper, we present i) a flexible event-driven platform consisting of the integration of a visual AER sensor and the SpiNNaker system, a programmable massively parallel digital architecture oriented to the simulation of spiking neural networks; ii) the implementation of a neural network for feature-based attentional selection on this platform.KeywordsAttentionSelectionNeuromorphicSpiNNakerAER

Variable behavior has been observed in several mechanisms found in biological neurons, resulting in changes in neural behavior that might be useful to capture in neuromorphic circuits. This paper presents a neuromorphic cortical neuron with synaptic neurotransmitter-release variability, which is designed to be used in neural networks as part of the Biomimetic Real-Time Cortex project. This neuron has been designed and simulated using carbon nanotube (CNT) transistors, which is one of several nanotechnologies under consideration to meet the challenges of scale presented by the cortex. Some research results suggest that some instances of variability are stochastic, while others indicate that some instances of variability are chaotic. In this paper, both possible sources of variability are considered by embedding either Gaussian noise or a chaotic signal into the neuromorphic or synaptic circuit and observing the simulation results. In order to embed chaotic behavior into the neuromorphic circuit, a chaotic signal generator circuit is presented, implemented with CNT transistors that could be embedded in the electronic neural circuit, and simulated using CNT SPICE models. The circuit uses a chaotic piecewise linear 1-D map implemented by switched-current circuits. The simulation results presented in this paper illustrate that neurotransmitter-release variability plays a beneficial role in the reliability of spike generation. In an examination of this reliability, the precision of spike timing in the CNT circuit simulations is found to be dependent on stimulus (postsynaptic potential) transients. Postsynaptic potentials with low neurotransmitter release variability or without neurotransmitter release variability produce imprecise spike trains, whereas postsynaptic potentials with high neurotransmitter-release variability produce spike trains with reproducible timing.

Spiking neural network simulations incorporating variable transmission delays require synaptic events to be scheduled prior to delivery. Conventional methods have memory requirements that scale with the total number of synapses in a network. We introduce novel scheduling algorithms for both discrete and continuous event delivery, where the memory requirement scales instead with the number of neurons. Superior algorithmic performance is demonstrated using large-scale, benchmarking network simulations.

This manuscript is focused on scalability of Spiking Neural Network (SNN) for acceleration of its learning time. Simulation of SNN algorithm was implemented on GPUs devices Ge Force 9400M and Ge Force650 GTX in order to compare the learning time. Multiclass database are used for classification and the results are compared.

Simulation of spiking neural networks — architectures and implementations

Event-driven simulation of neural population synchronization facilitated by electrical coupling