Inverse stochastic resonance in adaptive small-world neural networks.

Electromagnetic stimulation is an important neuromodulation technique that modulates the electrical activity of neurons and affects cortical excitability for the purpose of modulating the nervous system. The phenomenon of inverse stochastic resonance is a response mechanism of the biological nervous system to external signals and plays an important role in the signal processing of the nervous system. In this paper, a small-world neural network with electrical synaptic connections was constructed, and the inverse stochastic resonance of the small-world neural network under electromagnetic stimulation was investigated by analyzing the dynamics of the neural network. The results showed that: the Levy channel noise under electromagnetic stimulation could cause the occurrence of inverse stochastic resonance in small-world neural networks; the characteristic index and location parameter of the noise had significant effects on the intensity and duration of the inverse stochastic resonance in neural networks; the larger the probability of randomly adding edges and the number of nearest neighbor nodes in small-world networks, the more favorable the anti-stochastic resonance was; by adjusting the electromagnetic stimulation parameters, a dual regulation of the inverse stochastic resonance of the neural network can be achieved. The results of this study provide some theoretical support for exploring the regulation mechanism of electromagnetic nerve stimulation technology and the signal processing mechanism of nervous system.

Rats and mice use their whiskers to probe the environment. By rhythmically swiping their whiskers back and forth they can detect the existence of an object, locate it, and identify its texture. Localization can be accomplished by inferring the whisker’s position. Rhythmic neurons that track the phase of the whisking cycle encode information about the azimuthal location of the whisker. These neurons are characterized by preferred phases of firing that are narrowly distributed. Consequently, pooling the rhythmic signal from several upstream neurons is expected to result in a much narrower distribution of preferred phases in the downstream population, which however has not been observed empirically. Here, we show how spike timing dependent plasticity (STDP) can provide a solution to this conundrum. We investigated the effect of STDP on the utility of a neural population to transmit rhythmic information downstream using the framework of a modeling study. We found that under a wide range of parameters, STDP facilitated the transfer of rhythmic information despite the fact that all the synaptic weights remained dynamic. As a result, the preferred phase of the downstream neuron was not fixed, but rather drifted in time at a drift velocity that depended on the preferred phase, thus inducing a distribution of preferred phases. We further analyzed how the STDP rule governs the distribution of preferred phases in the downstream population. This link between the STDP rule and the distribution of preferred phases constitutes a natural test for our theory.

The problem with training spiking neural networks (SNNs) is relevant due to the ultra-low power consumption these networks could exhibit when implemented in neuromorphic hardware. The ongoing progress in the fabrication of memristors, a prospective basis for analogue synapses, gives relevance to studying the possibility of SNN learning on the base of synaptic plasticity models, obtained by fitting the experimental measurements of the memristor conductance change. The dynamics of memristor conductances is (necessarily) nonlinear, because conductance changes depend on the spike timings, which neurons emit in an all-or-none fashion. The ability to solve classification tasks was previously shown for spiking network models based on the bio-inspired local learning mechanism of spike-timing-dependent plasticity (STDP), as well as with the plasticity that models the conductance change of nanocomposite (NC) memristors. Input data were presented to the network encoded into the intensities of Poisson input spike sequences. This work considers another approach for encoding input data into input spike sequences presented to the network: temporal encoding, in which an input vector is transformed into relative timing of individual input spikes. Since temporal encoding uses fewer input spikes, the processing of each input vector by the network can be faster and more energy-efficient. The aim of the current work is to show the applicability of temporal encoding to training spiking networks with three synaptic plasticity models: STDP, NC memristor approximation, and PPX memristor approximation. We assess the accuracy of the proposed approach on several benchmark classification tasks: Fisher’s Iris, Wisconsin breast cancer, and the pole balancing task (CartPole). The accuracies achieved by SNN with memristor plasticity and conventional STDP are comparable and are on par with classic machine learning approaches.

Sensory systems pass information about an animal's environment to higher nervous system units through sequences of action potentials. When these action potentials have essentially equivalent wave forms, all information is contained in the interspike intervals (ISIs) of the spike sequence. How do neural circuits recognize and read these ISI sequences? We address this issue of temporal sequence learning by a neuronal system utilizing spike timing dependent plasticity (STDP). We present a general architecture of neural circuitry that can perform the task of ISI recognition. The essential ingredients of this neural circuit, which we refer to as "interspike interval recognition unit" (IRU) are (i) a spike selection unit, the function of which is to selectively distribute input spikes to downstream IRU circuitry; (ii) a time-delay unit that can be tuned by STDP; and (iii) a detection unit, which is the output of the IRU and a spike from which indicates successful ISI recognition by the IRU. We present two distinct configurations for the time-delay circuit within the IRU using excitatory and inhibitory synapses, respectively, to produce a delayed output spike at time t_{0}+tau(R) in response to the input spike received at time t_{0} . R is the tunable parameter of the time-delay circuit that controls the timing of the delayed output spike. We discuss the forms of STDP rules for excitatory and inhibitory synapses, respectively, that allow for modulation of R for the IRU to perform its task of ISI recognition. We then present two specific implementations for the IRU circuitry, derived from the general architecture that can both learn the ISIs of a training sequence and then recognize the same ISI sequence when it is presented on subsequent occasions.

This paper proposes a theoretical and experimental analysis of the expected running time for an elitist parallel Evolutionary Algorithm (pEA) based on an island model executed over small-world networks. Our study assumes the resolution of optimization problems based on unimodal pseudo-boolean funtions. In particular, for such function with d values, we improve the previous asymptotic upper bound for the expected parallel running time from O(d√n) to O(d log n). This study is a first step towards the analysis of influence of more complex network topologies (like random graphs created by P2P networks) on the runtime of pEAs. A concrete implementation of the analysed algorithm have been performed on top of the ParadisEO framework and run on the HPC platform of the University of Luxembourg (UL). Our experiments confirm the expected speedup demonstrated in this article and prove the benefit that pEA can gain from a small-world network topology.

Spike coherence and synchronization on Newman–Watts small-world neuronal networks modulated by spike-timing-dependent plasticity

Brain functions such as learning and memory rely on synaptic plasticity. Many studies have shown that synaptic plasticity can be driven by the timings between pre- and postsynaptic spikes, also known as spike-timing-dependent plasticity (STDP). In most of the modeling studies exploring STDP functions, presynaptic spikes have been postulated to be Poisson (random) spikes and postsynaptic neurons have been described using an integrate-and-fire model, for simplicity. However, experimental data suggest this is not necessarily true. In this study, we investigated how STDP worked in synaptic competition if more neurophysiologically realistic properties for pre- and postsynaptic dynamics were incorporated; presynaptic (input) spikes obeyed a gamma process and the postsynaptic neuron was a multi-timescale adaptive threshold model. Our results showed that STDP strengthened specific combinations of pre- and postsynaptic properties; regular spiking neurons favored regular input spikes whereas random spiking neurons did random ones, suggesting neural information coding utilizes both the properties.

One of the most important objectives of studying complex network is to reveal the dynamic behavior of network, to research the information transfer and to explore the influence of static or dynamic network structure on information transfer. This paper proposed a modified model, called the Swapped Transfer Model (STM) of information in complex network. The model formulates multiple aspects of information transfer and introduces the information growth factor. It simulates the information transfer in complex network based on ‘swapping’ ideas. Through the STM model, some special rules of information transfer are studied and analyzed thoroughly. Experiments show that information is transferring quickly in the small-world network. The transferring process is affected by different (internal and external) parameters (factors) which are examined through the dedicated parameters-separated experiments.

Spike timing dependent plasticity (STDP) learning rule is one of hot topics in neurobiology since it's been widely believed that synaptic plasticity mainly contribute to learning and memory in brain. Up to now, STDP has been observed in a wide variety of areas of brain, hippocampus, cortex and so on. Competition among synapses is an important behavior for this learning rule. In present study, we propose a single layer spiking neural network model using STDP learning rule in inhibitory synapses to investigate the competitive behavior. The experiments show that the synapses among neurons are both strengthened on the whole training process. Thus neurons inhibit the activities of one another, eventually the neuron with the highest input spike rate win the competition. We have found that the behavior is efficient when the differences of firing rates of input neurons without STDP are great than 5Hz, otherwise the winner neuron is random. In order to use the principle to artificial intelligent system, we use a mechanism of dynamic learning rate to let the neuron with the highest input to be selected by the competitive behavior as the winner. Therefore, a robust competitive spiking neural network is obtained.

Attitude interaction for financial price behaviours by contact system with small-world network topology

<i>In vitro</i> electrophysiological investigation of neural activity at a network level holds tremendous potential for elucidating underlying features of brain function (and dysfunction). In standard neural network modelling systems, however, the fundamental three-dimensional (3D) character of the brain is a largely disregarded feature. This widely applied neuroscientific strategy affects several aspects of the structure–function relationships of the resulting networks, altering network connectivity and topology, ultimately reducing the translatability of the results obtained. As these model systems increase in popularity, it becomes imperative that they capture, as accurately as possible, fundamental features of neural networks in the brain, such as small-worldness. In this report, we combine <i>in vitro</i> neural cell culture with a biologically compatible scaffolding substrate, surface-grafted polymer particles (PPs), to develop neural networks with 3D topology. Furthermore, we investigate their electrophysiological network activity through the use of 3D multielectrode arrays. The resulting neural network activity shows emergent behaviour consistent with maturing neural networks capable of performing computations, i.e. activity patterns suggestive of both information segregation (desynchronized single spikes and local bursts) and information integration (network spikes). Importantly, we demonstrate that the resulting PP structured neural networks show both structural and functional features consistent with small-world network topology.

This work investigates elementary cellular automata comparing their dynamic evolution on two topologies: the usual regular lattice and a small-world network constructed as a directed graph. The number of iterations for any initial perturbation to propagate over all cells of the automaton corresponds to how quickly an initial configuration reaches a high entropy state, and the temporal rate of propagation of the perturbation can be evaluated from a chaotic set of elementary cellular automata rules using an entropy measure. Results indicate that the average entropy can be nearly tripled in the small-world network topology, suggesting much faster applications (e.g. in Cryptography) from alterations in the topological arrangement of usual cellular automata regular lattices.

Pattern synchronization in a two-layer neuronal network is studied. For a single-layer network of Rulkov map neurons, there are three kinds of patterns induced by noise. Additive noise can induce ordered patterns at some intermediate noise intensities in a resonant way; however, for small and large noise intensities there exist excitable patterns and disordered patterns, respectively. For a neuronal network coupled by two single-layer networks with noise intensity differences between layers, we find that the two-layer network can achieve synchrony as the interlayer coupling strength increases. The synchronous states strongly depend on the interlayer coupling strength and the noise intensity difference between layers.

Spiking neural networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. The challenge of spike-based encoding has been the lack of a universal training mechanism performed entirely using spikes. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANNs): (1) SNNs can be trained by externally computed numerical gradients. (2) A major advancement towards native spike-based learning has been the use of approximate BP using spike-time dependent plasticity with phased forward/backward passes. However, the transfer of information between such phases for gradient and weight update calculation necessitates external memory and computational access. This is a challenge for standard neuromorphic hardware implementations. In this paper, we propose a stochastic SNN based back-prop (SSNN-BP) algorithm that utilizes a composite neuron to simultaneously compute the forward pass activations and backward pass gradients explicitly with spikes. Although signed gradient values are a challenge for spike-based representation, we tackle this by splitting the gradient signal into positive and negative streams. The composite neuron encodes information in the form of stochastic spike-trains and converts BP weight updates into temporally and spatially local spike coincidence updates compatible with hardware-friendly resistive processing units. Furthermore, we characterize the quantization effect of discrete spike-based weight update to show that our method approaches BP ANN baseline with sufficiently long spike-trains. Finally, we show that the well-performing softmax cross-entropy loss function can be implemented through inhibitory lateral connections enforcing a winner take all rule. Our SNN with a two-layer network shows excellent generalization through comparable performance to ANNs with equivalent architecture and regularization parameters on static image datasets like MNIST, Fashion-MNIST, Extended MNIST, and temporally encoded image datasets like Neuromorphic MNIST datasets. Thus, SSNN-BP enables BP compatible with purely spike-based neuromorphic hardware.

Sparsely-synchronized cortical rhythms, associated with diverse cognitive functions, have been observed in electric recordings of brain activity. At the population level, cortical rhythms exhibit small-amplitude fast oscillations while at the cellular level, individual neurons show stochastic firings sparsely at a much lower rate than the population rate. We study the effect of network architecture on sparse synchronization in an inhibitory population of subthreshold Morris-Lecar neurons (which cannot fire spontaneously without noise). Previously, sparse synchronization was found to occur for cases of both global coupling (i.e., regular all-to-all coupling) and random coupling. However, a real neural network is known to be non-regular and non-random. Here, we consider sparse Watts-Strogatz small-world networks which interpolate between a regular lattice and a random graph via rewiring. We start from a regular lattice with only short-range connections and then investigate the emergence of sparse synchronization by increasing the rewiring probability p for the short-range connections. For p = 0, the average synaptic path length between pairs of neurons becomes long; hence, only an unsynchronized population state exists because the global efficiency of information transfer is low. However, as p is increased, long-range connections begin to appear, and global effective communication between distant neurons may be available via shorter synaptic paths. Consequently, as p passes a threshold p th (}~ 0.044), sparsely-synchronized population rhythms emerge. However, with increasing p, longer axon wirings become expensive because of their material and energy costs. At an optimal value p* DE (}~ 0.24) of the rewiring probability, the ratio of the synchrony degree to the wiring cost is found to become maximal. In this way, an optimal sparse synchronization is found to occur at a minimal wiring cost in an economic small-world network through trade-off between synchrony and wiring cost.