Efficient implementation of STDP rules on SpiNNaker neuromorphic hardware

A forecast-based STDP rule suitable for neuromorphic implementation

Since the discovery of place cells – single pyramidal neurons that encode spatial location – it has been hypothesized that the hippocampus may act as a cognitive map of known environments. This putative function has been extensively modeled using auto-associative networks, which utilize rate-coded synaptic plasticity rules in order to generate strong bi-directional connections between concurrently active place cells that encode for neighboring place fields. However, empirical studies using hippocampal cultures have demonstrated that the magnitude and direction of changes in synaptic strength can also be dictated by the relative timing of pre- and post-synaptic firing according to a spike-timing dependent plasticity (STDP) rule. Furthermore, electrophysiology studies have identified persistent “theta-coded” temporal correlations in place cell activity in vivo, characterized by phase precession of firing as the corresponding place field is traversed. It is not yet clear if STDP and theta-coded neural dynamics are compatible with cognitive map theory and previous rate-coded models of spatial learning in the hippocampus. Here, we demonstrate that an STDP rule based on empirical data obtained from the hippocampus can mediate rate-coded Hebbian learning when pre- and post-synaptic activity is stochastic and has no persistent sequence bias. We subsequently demonstrate that a spiking recurrent neural network that utilizes this STDP rule, alongside theta-coded neural activity, allows the rapid development of a cognitive map during directed or random exploration of an environment of overlapping place fields. Hence, we establish that STDP and phase precession are compatible with rate-coded models of cognitive map development.

Spike-timing dependent plasticity (STDP), a synaptic modification depending on a relative timing of presynaptic and postsynaptic spikes, has fascinated researchers in the fields of neurophysiology and computational neuroscience, because it is not only conceptually simple or biologically reasonable but is also versatile in neural network simulations. The STDP rule may be valid only under specific conditions, however. We propose herein a method that could find more natural and potent rules of synaptic plasticity.

Rats and mice use their whiskers to probe the environment. By rhythmically swiping their whiskers back and forth they can detect the existence of an object, locate it, and identify its texture. Localization can be accomplished by inferring the whisker’s position. Rhythmic neurons that track the phase of the whisking cycle encode information about the azimuthal location of the whisker. These neurons are characterized by preferred phases of firing that are narrowly distributed. Consequently, pooling the rhythmic signal from several upstream neurons is expected to result in a much narrower distribution of preferred phases in the downstream population, which however has not been observed empirically. Here, we show how spike timing dependent plasticity (STDP) can provide a solution to this conundrum. We investigated the effect of STDP on the utility of a neural population to transmit rhythmic information downstream using the framework of a modeling study. We found that under a wide range of parameters, STDP facilitated the transfer of rhythmic information despite the fact that all the synaptic weights remained dynamic. As a result, the preferred phase of the downstream neuron was not fixed, but rather drifted in time at a drift velocity that depended on the preferred phase, thus inducing a distribution of preferred phases. We further analyzed how the STDP rule governs the distribution of preferred phases in the downstream population. This link between the STDP rule and the distribution of preferred phases constitutes a natural test for our theory.

Spike-timing-dependent plasticity (STDP) is considered as a primary mechanism underlying formation of new memories during learning. Despite the growing interest in activity-dependent plasticity, it is still unclear whether synaptic plasticity rules inferred from in vitro experiments are correct in physiological conditions. The abnormally high calcium concentration used in in vitro studies of STDP suggests that in vivo plasticity rules may differ significantly from in vitro experiments, especially since STDP depends strongly on calcium for induction. We therefore studied here the influence of extracellular calcium on synaptic plasticity. Using a combination of experimental (patch-clamp recording and Ca2+ imaging at CA3-CA1 synapses) and theoretical approaches, we show here that the classic STDP rule in which pairs of single pre- and postsynaptic action potentials induce synaptic modifications is not valid in the physiological Ca2+ range. Rather, we found that these pairs of single stimuli are unable to induce any synaptic modification in 1.3 and 1.5 mM calcium and lead to depression in 1.8 mM. Plasticity can only be recovered when bursts of postsynaptic spikes are used, or when neurons fire at sufficiently high frequency. In conclusion, the STDP rule is profoundly altered in physiological Ca2+, but specific activity regimes restore a classical STDP profile.

Emerging nonvolatile memory technologies have been much studied to realize massively interconnected Spiking Neural Networks (SNNs) due to its high density, and energy efficiency. Nevertheless, most of the previous studies on utilizing such technologies have focused on implementing a traditional pair-based Spike Timing Dependent Plasticity (STDP) rule, which turned out to fail to reproduce multiple physiological experimental results. In this paper, we present a neuromorphic circuit for a three-terminal spintronic synapse with a triplet-based STDP rule, which is a higher order of learning mechanism. Simulation results indicate that the proposed learning circuit for the triplet-based STDP rule significantly improves learning capability in mimicking the various biological experimental data. We introduce a normalized mean-square error $\boldsymbol{E}$ value to evaluate the performance of each of the learning circuits quantitatively. The proposed learning circuit achieves $\boldsymbol{E}$ value of 1.77, which is far better than the conventional one that achieves $\boldsymbol{E}$ value of 12.2.

The influence of a weight-dependent spike-timing dependent plasticity (STDP) rule on the temporal evolution and equilibrium state of a certain synapse is investigated. We show that under certain conditions, a spike-induced rate-learning scheme could be achieved. Through studying the situation when a single Hodgkin-Huxley neuron is driven by a large ensemble of input neurons, we find that synchronized firing of a sub population of input neurons may be important to information processing in the nervous system. Using simulations, we show that the temporal structure of the spike trains of these synchronized input neurons can be transmitted reliably; further, synapses from these neurons will increase stably due to the STDP rule and this may provide a mechanism for learning and information storage in biologically plausible network models.

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

The problem with training spiking neural networks (SNNs) is relevant due to the ultra-low power consumption these networks could exhibit when implemented in neuromorphic hardware. The ongoing progress in the fabrication of memristors, a prospective basis for analogue synapses, gives relevance to studying the possibility of SNN learning on the base of synaptic plasticity models, obtained by fitting the experimental measurements of the memristor conductance change. The dynamics of memristor conductances is (necessarily) nonlinear, because conductance changes depend on the spike timings, which neurons emit in an all-or-none fashion. The ability to solve classification tasks was previously shown for spiking network models based on the bio-inspired local learning mechanism of spike-timing-dependent plasticity (STDP), as well as with the plasticity that models the conductance change of nanocomposite (NC) memristors. Input data were presented to the network encoded into the intensities of Poisson input spike sequences. This work considers another approach for encoding input data into input spike sequences presented to the network: temporal encoding, in which an input vector is transformed into relative timing of individual input spikes. Since temporal encoding uses fewer input spikes, the processing of each input vector by the network can be faster and more energy-efficient. The aim of the current work is to show the applicability of temporal encoding to training spiking networks with three synaptic plasticity models: STDP, NC memristor approximation, and PPX memristor approximation. We assess the accuracy of the proposed approach on several benchmark classification tasks: Fisher’s Iris, Wisconsin breast cancer, and the pole balancing task (CartPole). The accuracies achieved by SNN with memristor plasticity and conventional STDP are comparable and are on par with classic machine learning approaches.

The liquid state machine (LSM) is a model of recurrent spiking neural networks (SNNs) and provides an appealing brain-inspired computing paradigm for machine-learning applications. Moreover, operated by processing information directly on spiking events, the LSM is amenable to efficient event-driven hardware implementation. However, training SNNs is, in general, a difficult task as synaptic weights shall be updated based on neural firing activities while achieving a learning objective. In this article, we explore bio-plausible spike-timing-dependent-plasticity (STDP) mechanisms to train liquid state machine models with and without supervision. First, we employ a supervised STDP rule to train the output layer of the LSM while delivering good classification performance. Furthermore, a hardware-friendly unsupervised STDP rule is leveraged to train the recurrent reservoir to further boost the performance. We pursue efficient hardware implementation of FPGA LSM accelerators by performing algorithm-level optimization of the two proposed training rules and exploiting the self-organizing behaviors naturally induced by STDP. Several recurrent spiking neural accelerators are built on a Xilinx Zync ZC-706 platform and trained for speech recognition with the TI46 speech corpus as the benchmark. Adopting the two proposed unsupervised and supervised STDP rules outperforms the recognition accuracy of a competitive non-STDP baseline training algorithm by up to 3.47%.

Neuromorphic engineering has a wide range of applications in the fields of machine learning, pattern recognition, adaptive control, etc. Photonics, characterized by its high speed, wide bandwidth, low power consumption and massive parallelism, is an ideal way to realize ultrafast spiking neural networks (SNNs). Synaptic plasticity is believed to be critical for learning, memory and development in neural circuits. Experimental results have shown that changes of synapse are highly dependent on the relative timing of pre- and postsynaptic spikes. Synaptic plasticity in which presynaptic spikes preceding postsynaptic spikes results in strengthening, while the opposite timing results in weakening is called antisymmetric spike-timing-dependent plasticity (STDP) learning rule. And synaptic plasticity has the opposite effect under the same conditions is called antisymmetric anti-STDP learning rule. We proposed and experimentally demonstrated an optical implementation of neural learning algorithms, which can achieve both of antisymmetric STDP and anti-STDP learning rule, based on the cross-gain modulation (XGM) within a single semiconductor optical amplifier (SOA). The weight and height of the potentitation and depression window can be controlled by adjusting the injection current of the SOA, to mimic the biological antisymmetric STDP and anti-STDP learning rule more realistically. As the injection current increases, the width of depression and potentitation window decreases and height increases, due to the decreasing of recovery time and increasing of gain under a stronger injection current. Based on the demonstrated optical STDP circuit, ultrafast learning in optical SNNs can be realized.

BackgroundAssociative high-frequency electrical stimulation (HFS) of the supraorbital nerve in five healthy individuals induced long-term potentiation (LTP)-like or depression (LTD)-like changes in the human blink reflex circuit according to the rules of spike timing-dependent plasticity (Mao and Evinger, 2001). HFS given at the onset of the R2 component of the blink reflex (HFSLTP) produced a lasting facilitation of the R2, whereas HFS given shortly before R2 (HFSLTD) caused a lasting suppression of the R2. In patients with benign essential blepharospasm (BEB), a focal dystonia affecting the orbicularis oculi muscles, HFSLTP induced excessive LTP-like associative plasticity relative to healthy controls, which was normalized after botulinum toxin (BTX) injections (Quartarone et al, 2006).Methodology/Principal FindingsWe used HFS conditioning of the supraorbital nerve to study homeostatic metaplasticity of the blink reflex circuit in healthy subjects and dystonic patients. On separate days, we tested the conditioning effects on the R2 response and paired-pulse R2 inhibition after (i) HFSLTP, (ii) HFSLTP followed by HFSLTP, and (iii) HFSLTP followed by HFSLTD. Controls also received (iv) HFSLTD alone and (v) a non-intervention protocol. In BEB patients, HFSLTP followed by HFSLTD was given before and after BTX treatment. We were not able to replicate the bidirectional timing-dependent effects of HFSLTP and HFSLTD alone. All HFS protocols produced a non-specific reduction of the R2 response and a relative decrease in paired-pulse inhibition. These R2 changes also occurred in controls when no HFS was applied. There was also no trace of a homeostatic response pattern in BEB patients before or after BTX treatment.Conclusion/SignificanceOur data challenge the efficacy of associative HFS to produce bidirectional plasticity in the human blink reflex circuit. The non-specific decrease of the R2 response might indicate habituation of the blink reflex following repeated electrical supraorbital stimulation. The increase of inhibition after paired pulse stimulation might reflect homeostatic behaviour to prevent further down regulation of the R2 response to preserve the protection of this adverse-effects reflex.

The spike-timing-dependent-plasticity (STDP) is considered to be the basic function for the synapse device to simulate biological activities of the brain. In this paper, we demonstrate four standard STDP forms with nanosecond timescale based on the analog property of RRAM. An operation scheme to achieve diverse STDP rules is proposed according to the relationship between conductance change and the applied voltage of pulses. The diverse ultra-fast STDP rules are realized experimentally in fabricated Al2O3/ HfO2-based RRAM. The maximum weight change is up to 300% under nanosecond timescale pulses. Moreover, the nonvolatile characteristic of the device enables long-term potentiation and depression.

Action Potential (APs) patterns of sensory cortex neurons encode a variety of stimulus features, but how can a neuron change the feature to which it responds? Here, we show that in vivo a spike-timing-dependent plasticity (STDP) protocol-consisting of pairing a postsynaptic AP with visually driven presynaptic inputs-modifies a neurons' AP-response in a bidirectional way that depends on the relative AP-timing during pairing. Whereas postsynaptic APs repeatedly following presynaptic activation can convert subthreshold into suprathreshold responses, APs repeatedly preceding presynaptic activation reduce AP responses to visual stimulation. These changes were paralleled by restructuring of the neurons response to surround stimulus locations and membrane-potential time-course. Computational simulations could reproduce the observed subthreshold voltage changes only when presynaptic temporal jitter was included. Together this shows that STDP rules can modify output patterns of sensory neurons and the timing of single-APs plays a crucial role in sensory coding and plasticity.DOI:http://dx.doi.org/10.7554/eLife.00012.001.

Efficient Spike Timing Dependent Plasticity rule for Complex-Valued Neurons