Associative memory of weakly connected oscillators

This paper presents the results of a study of synchronization processes in oscillatory neural networks of various structures based on the Kuramoto model. The estimates of synchronization processes occurring in the oscillatory networks are examined. The results of studying the practical application of oscillatory networks for solving cluster analysis problems are presented.

A study on multi-frequency patterns in nonlinear network oscillators using incremental harmonic balance method

Oscillatory networks are a special class of neural networks where each neuron exhibits time periodic behavior. They represent bio-inspired architectures which can be exploited to model biological processes such as the binding problem and selective attention. In most of situations, each neuron is assumed to have a stable limit cycle as the unique attractor. In this paper we investigate the dynamics of networks whose neurons are hard oscillators, namely they exhibit the coexistence of a stable limit cycle and a stable equilibrium point. We consider a constant external stimulus applied to each neuron, which influences the neuron's own natural frequency. We investigate the bifurcations in the neuron's dynamics induced by the input. We show that, due to the interaction between different kind of attractors, as well as between attractors and repellors, new interesting dynamics arises, in the form of synchronous oscillations of various amplitudes. We also show that neurons subject to different stimuli are able to synchronize if their couplings are strong enough.

Both time delay and coupling form are the most important factors in neural networks. The properties of firing rate oscillation and cluster synchronization induced by time delay are studied in random network of different coupling neurons. In previous work, the firing rate oscillation of cortical network was observed at the presence of three factors (time delay, weak sinusoidal signal, and noise). Here, we found that the firing rate oscillation can be induced only by the time delay, and the spike train can be propagated at a certain interval time, which is consistent with the value of delay time. Furthermore, the phenomenon of cluster synchronization occurs in random network, which may originates from network structure, and this connection between the neurons trigger spikes within a time-restricted window, resulting in cluster synchronization between corresponding neurons. These numerical results provide a potential theoretical basis for certain pathological brain rhythms associated with epileptic seizures.

The Sixth International Symposium on Neural Networks (ISNN 2009)

The growing number of edge devices in everyday life generates a considerable amount of data that current AI algorithms, like artificial neural networks, cannot handle inside edge devices with limited bandwidth, memory, and energy available. Neuromorphic computing, with low-power oscillatory neural networks (ONNs), is an alternative and attractive solution to solve complex problems at the edge. However, ONN is currently limited with its fully-connected recurrent architecture to solve auto-associative memory problems. In this work, we use an alternative two-layer bidirectional ONN architecture. We introduce a two-layer feedforward ONN architecture to perform image edge detection, using the ONN to replace convolutional filters to scan the image. Using an HNN Matlab emulator and digital ONN design simulations, we report efficient image edge detection from both architectures using various size filters (3 × 3, 5 × 5, and 7 × 7) on black and white images. In contrast, the feedforward architectures can also perform image edge detection on gray scale images. With the digital ONN design, we also assess latency performances and obtain that the bidirectional architecture with a 3 × 3 filter size can perform image edge detection in real-time (camera flow from 25 to 30 images per second) on images with up to 128 × 128 pixels while the feedforward architecture with same 3 × 3 filter size can deal with 170 × 170 pixels, due to its faster computation.

Self-sustained oscillatory dynamics is a motion along a stable limit cycle in the phase space, and it arises in a wide variety of mechanical, electrical, and biological systems. Typically, oscillations are due to a balance between energy dissipation and generation. Their stability depends on the properties of the attractor, in particular, its dissipative characteristics, which in turn determine the flexibility of a given dynamical system. In a network of oscillators, the coupling additionally contributes to the dissipation, and hence affects the robustness of the oscillatory solution. Here, we therefore investigate how a heterogeneous network structure affects the dissipation rate of individual oscillators. First, we show that in a network of diffusively coupled oscillators, the dissipation is a linearly decreasing function of the node degree, and we demonstrate this numerically by calculating the average divergence of coupled Hopf oscillators. Subsequently, we use recordings of intracellular calcium dynamics in pancreatic beta cells in mouse acute tissue slices and the corresponding functional connectivity networks for an experimental verification of the presented theory. We use methods of nonlinear time series analysis to reconstruct the phase space and calculate the sum of Lyapunov exponents. Our analysis reveals a clear tendency of cells with a higher degree, that is, more interconnected cells, having more negative values of divergence, thus confirming our theoretical predictions. We discuss these findings in the context of energetic aspects of signaling in beta cells and potential risks for pathological changes in the tissue.

Information processing and two types of memory in an analog neural network model with time delay that produces chaos similar to the human and animal EEGs are considered. There are two levels of information processing in this neural network: the level of individual neurons and the level of the neural network. Similar to the state of brain, the state of chaotic neural network is defined. It is characterized by two types of memories (memory I and memory II) and correlation structure between the neurons. In normal (unperturbed) state, the neural network generates chaotic patterns of averaged neuronal activities (memory I) and patterns of oscillation amplitudes (memory II). In the presence of external stimulation, the activity patterns change, showing changes in both types of memory. As in experiments on stimulation of the brain, the neural network model shows synchronization of neuronal activities due to stimulus measured by Pearson's correlation coefficient. An increase in neural network asymmetry (increase of the neural network excitability) leads to the phenomenon similar to the epilepsy. Modeling of brain injury, Parkinson's disease, and dementia is performed by removing and weakening interneuron connections. In all cases, the chaotic neural network shows a decrease of the degree of chaos and changes in both types of memory similar to those observed in experiments with healthy human subjects and patients with Parkinson's disease and dementia. © 2005 Wiley Periodicals, Inc. Complexity 11:39–52, 2005

Pattern recognition minimizes entropy production in a neural network of electrical oscillators

The investigation of nonlinear processes in brain systems based on the oscillatory-wave approach is currently one of the most consequential domains of researching signal generation and processing mechanisms in the brain. These investigations may offer insights into various phenomena, such as the mechanism of associative memory, based on network models of biologically relevant neurons. A neural network architecture is proposed in this paper to solve applied problems, such as signal filtering, processing, and recognition of information images. Mechanisms for incomplete fragments were implemented, enabling the extraction and complete restoration of objects from memory. Each neuron in this neural network comprises a Hodgkin–Huxley biophysical model with a Mainen modification, the dynamics of which most plausibly reproduce the processes in the neural cells of the brain. In the research on individual neurons, a constant external current with varying amplitudes was applied. The study determined the average periods of self-oscillations and identified parameter values for the Andronov–Hopf bifurcation. In particular, a stable limit cycle is created and destroyed depending on different scenarios when the bias current is increased or decreased. A stable limit cycle arises via the Andronov–Hopf bifurcation and is terminated via the saddle-node bifurcation on the cycle. The study identified parameter regions whereby two neurons with excitatory and inhibitory synaptic connections synchronize. The neurons were all in self-oscillating mode and exhibited a stable limit cycle in the phase space. A three-layer neural network was constructed, comprising of a reference neuron, sensory, control, and interneurons. The control and intermediate layer connections are arranged based on the Hebbian learning rule. Areas of excitation for various types of neurons were found. Experiments were conducted using the neural network architecture to distinguish binary information patterns encoded through signal phase. Excitatory and inhibitory synaptic connections were employed to encode the images stored in memory. The dynamic patterns are determined through the in-phase or anti-phase mode of phase locking, i.e., phase synchronization relative to the global rhythm. Before reaching the output layer of neurons, the signals are summed on the interneuron layer with the signal of the reference neuron, resulting in the filtering out of a certain information segment (in-phase or anti-phase). The network accurately recognized the patterns after parameter adjustments were made. Areas of neuron phase synchronization were discovered, enabling control of network activity modes. The confirmation of the viability of phase capture and pattern recognition modes using Hodgkin–Huxley–Mainen neurons was achieved. In total, as a result of the implementation of the spike neural network modeling project: 1) two-parameter diagrams of regions of neuronal excitation in various modes are calculated; 2) the effect of excitatory and inhibitory coupling in synaptic currents is used to represent the input signal; 3) an abstract mathematical algorithm for calculating the Hebbian matrix has been transferred to the implementation of synaptic currents between layers of neurons; 4) the effect of phase clusters is used to represent the output pattern; 5) the neurons’ topology in the visual-cerebral department served as a working model for recognizing graphic images through the Hopfield network on a spike neural network, using the Hebbian rule.

Networks of coupled periodic oscillators (similar to the Kuramoto model) have been proposed as models of associative memory. However, error-free retrieval states of such oscillatory networks are typically unstable, resulting in a near zero capacity. This puts the networks at disadvantage as compared with the classical Hopfield network. Here we propose a simple remedy for this undesirable property and show rigorously that the error-free capacity of our oscillatory, associative-memory networks can be made as high as that of the Hopfield network. They can thus not only provide insights into the origin of biological memory, but can also be potentially useful for applications in information science and engineering.

The role of network structure and time delay in a metapopulation Wilson--Cowan model

Oscillatory neural networks are a relatively recent approach for the problem of image segmentation. Inside of this context, the oscillator neuron of Terman-Wang is presented, which is used as base element of an oscillatory network called LEGION (Locally Excitatory Globally Inhibitory Oscillator Network). The continuous version of the LEGION network, based on a set of differential equations, presents high computational complexity and has limited capacity of segmentation, which restricts its practical application, being adequate for implementation in parallel hardware topologies. To reduce the computational complexity in serial computers, an algorithm proposed by Terman and Wang is presented, which implies significant gain of speed in comparison to the continuous version and, in contrast, capacity to discriminate a unlimited number of segments. An interactive version of this algorithm was proposed and the results obtained in segmentation of transmission electron microscopy (TEM) images were evaluated, with the objective of measuring helium bubbles in silicon samples. As final result we found that the LEGION network presents itself as a singular alternative to solve problems of image segmentation, which provides simultaneously both spatial and temporal discrimination of segments.

Effects of the parameters on the oscillation frequency of Izhikevich spiking neural networks

A systematic and straightforward methodology to analyze and quench the friction-induced limit cycle conditions in electro-hydraulic servovalve control systems with interval transport lag is developed in this paper. The nonlinearity is linearized by its corresponding describing function. The delay time in transmission line is considered. The stability-equation method accompanied with parameter plane method provides a useful tool for the establishment of necessary conditions to sustain a limit cycle in the constructed limit cycle surface. The portrayed surface characterizes clear relationship among limit cycle amplitude, frequency, transport delay, and the controller coefficients to be designed. The asymptotic stability region, the unstable region, and the limit cycle region are identified in the parameter plane. The variation of the transport lag which could accelerate the generation of limit cycles is investigated. A feasible stable region is characterized in the parameter plane to allow flexible choice of controller gains. The quenching of limit cycle is achieved by selecting controller gains from the asymptotic stability region. It is seen that the friction-induced limit cycles can be effectively quenched via the tuning of the controller gains.