Pattern recognition minimizes entropy production in a neural network of electrical oscillators

Synchronization is crucial for the correct functionality of many natural and man-made complex systems. In this work we characterize the formation of synchronization patterns in networks of Kuramoto oscillators. Specifically, we reveal conditions on the network weights, structure and on the oscillators' natural frequencies that allow the phases of a group of oscillators to evolve cohesively, yet independently from the phases of oscillators in different clusters. Our conditions are applicable to general directed and weighted networks of heterogeneous oscillators. Surprisingly, although the oscillators exhibit nonlinear dynamics, our approach relies entirely on tools from linear algebra and graph theory. Further, we develop a control mechanism to determine the smallest (as measured by the Frobenius norm) network perturbation to ensure the formation of a desired synchronization pattern. Our procedure allows us to constrain the set of edges that can be modified, thus enforcing the sparsity structure of the network perturbation. The results are validated through a set of numerical examples.

Alternative paradigms to the von Neumann computing scheme are currently arousing huge interest. Oscillatory neural networks (ONNs) using emerging phase-change materials like VO2 constitute an energy-efficient, massively parallel, brain-inspired, in-memory computing approach. The encoding of information in the phase pattern of frequency-locked, weakly coupled oscillators makes it possible to exploit their rich non-linear dynamics and their synchronization phenomena for computing. A single fully connected ONN layer can implement an auto-associative memory comparable to that of a Hopfield network, hence Hebbian learning rule is the most widely adopted method for configuring ONNs for such applications, despite its well-known limitations. An extensive amount of literature is available about learning in Hopfield networks, with information regarding many different learning algorithms that perform better than the Hebbian rule. However, not all of these algorithms are useful for ONN training due to the constraints imposed by their physical implementation. This paper evaluates different learning methods with respect to their suitability for ONNs. It proposes a new approach, which is compared against previous works. The proposed method has been shown to produce competitive results in terms of pattern recognition accuracy with reduced precision in synaptic weights, and to be suitable for online learning.

Twisted tri-lobed tubes have garnered attention due to their exceptional heat transfer efficiency and straightforward fabrication. Existing literature lacks comprehensive assessments of the overall heat transfer performance of twisted tri-lobed tubes from the perspective of energy loss and irreversibility. This research aims to investigate entropy generation during turbulent water flow within twisted tri-lobed tubes, examining the influence of geometric parameters on local and average entropy production. Findings indicate that larger small circle radius (r) and straight lengths (l), coupled with smaller transition circle radius (R) and twist pitch lengths (p), result in diminished local and average heat transfer entropy production while enhancing local and average frictional entropy production, with heat transfer entropy generation dominating the overall entropy production. Additionally, with increasing Reynolds numbers, all twisted tubes demonstrate an increasing trend in average frictional entropy production, except for some cases (Case 1-Case 4) that exhibit an initial rise followed by a decline in average heat transfer entropy generation. Among the examined Reynolds range, Case 4 displays lower overall irreversibility compared to a plain tube. Following the second law of thermodynamics, Case 4 is preferred. The findings and methodology contribute to enhancing the thermodynamic evaluation of convective heat transfer in twisted tri-lobed tubes.

Entropy production of emerging turbulent scales in a temporal supercritical n-heptane/nitrogen three-dimensional mixing layer

Background and Objectives: In this paper, we consider the general case of the establishment/destruction of a completely synchronous state of phase oscillator networks with topologies of links such as “ring” and “small world”. The natural frequencies of the node oscillators are supposed to be distributed along the network according to an arbitrary law. Materials and Methods: The network of Kuramoto oscillators, consisting of 1000 node elements, was considered as the system under study theoretically and numerically. Results: The influence of quantitative characteristics of the dependence of natural frequencies on the number (spatial coordinate) of the oscillator on the onset of a completely synchronous state of the network of phase oscillators as well as the mechanism of the transition to a completely synchronous regime has been studied. An analytical expression forthe critical value ofthe coupling parameter correspondingtothe establishment of a fully synchronous regime within the network under consideration has been deduced. The theoretical results obtained have been compared with the results of the direct numerical simulation of the oscillator network behavior, with the excellent agreement being observed. Conclusion: It has been found that the dependence of the natural frequencies of oscillators on the spatial coordinate (or, on the number of the oscillator in the network) in the case of networks with the topology of links such as “ring” and “small world” determines completely the properties of such networks from the point of view of establishing the phase synchronization. Having based on the “ring” and “small world” network properties, it is possible to solve not only the problem of finding the critical value of the coupling parameter for the known frequency dependences on the coordinate, but also the problem of synthesizing such networks with the predetermined properties.

We investigate the limit sets of a network of coupled Kuramoto oscillators with a coupling matrix determined by a Hebb rule. These limit sets are the output of the network if used for the recognition of a defective binary pattern out of several given patterns, with the output pattern encoded in the oscillators' phases. We show that if all pairs of given patterns have maximum Hamming distance, there exists a degenerate attractive limit set that contains the steady states corresponding to each of the given patterns. As a result, switching between output patterns occurs for arbitrarily small modifications of the dynamics (for example, due to frequency inaccuracies). Even if the maximum Hamming distance constraint is dropped, numerical results suggest that the structural instability of the vector field persists. We conclude that the unique interchangeability of output patterns in Hebbian networks of Kuramoto oscillators, while sacrificing robustness, makes these networks more flexible than similar neural networks with separated, attractive output states.

Das Kuramoto-Modell mit Trägheit ist ein bekanntes und einfaches Modell für die Frequenzdynamik in Stromnetzen. Es wurde häufig genutzt um die Stabilität und Dynamik des Systems nahe dem vollständig synchronisierten Zustand zu untersuchen. Mit steigenden Anteil erneuerbarer Energiequellen und der damit verbundenen verminderten Schwungmasse im System werden Stromnetze jedoch anfälliger für das Auftreten unerwünschter dynamischer Zustände und großer Frequenzschwankungen. Das Ziel dieser Arbeit ist es, das dynamische Modell aus der Perspektive der nichtlinearen Dynamik zu untersuchen und die verschiedenen asymptotischen dynamischen Zustände neben synchronen Fixpunkt zu analysisieren. Es werden analytische und numerische Bedingungen für die Existenz solitärer Zustände einzelner Knoten sowie für die Koexistenz synchronisierter Cluster hergeleitet.

The Kuramoto model in complex networks

A biologically plausible model of myelin plasticity demonstrates how oligodendrocytes can selectively adjust conduction delays in axon bundles to allow robust communication between brain regions through long-range synchronization.

It is a well-known fact that oscillatory networks can operate as Hopfield-like neural networks, the only difference being that their attractors are limit cycles: one for each memorized pattern. The neuron activities are synchronized on the limit cycles, and neurons oscillate with fixed phase differences (time delays). We prove that this property is a natural attribute of general weakly connected neural networks, and it is relatively independent of the equations that describe the network activity. In particular, we prove an analogue of the Cohen-Grossberg convergence theorem for oscillatory neural networks.

The rate of entropy production due to chemical reaction is calculated for various combinations of parameter values in the cubic autocatalator model in an isothermal, continuous stirred tank reactor (CSTR) proposed by Gray and Scott and by Escher and Ross. Values of the entropy production averaged over periods of limit cycle oscillations are compared with those in coexistent unstable stationary states. It is found that in ranges of the residence time over which there are limit cycles, the entropy production in coexisting stationary states increases as the residence time is shortened, i.e., as the system is removed farther from thermodynamic equilibrium. The average entropy production over a limit cycle is less than that in the corresponding stationary state over wide ranges of parameter values, but not necessarily for the whole oscillatory region. More specifically, the former inequality always prevails in ranges where the entropy production of stationary states is larger, i.e., the residence time is shorter, but in some cases the inequality is reversed in ranges of lower magnitudes of the entropy production.

The investigation of nonlinear processes in brain systems based on the oscillatory-wave approach is currently one of the most consequential domains of researching signal generation and processing mechanisms in the brain. These investigations may offer insights into various phenomena, such as the mechanism of associative memory, based on network models of biologically relevant neurons. A neural network architecture is proposed in this paper to solve applied problems, such as signal filtering, processing, and recognition of information images. Mechanisms for incomplete fragments were implemented, enabling the extraction and complete restoration of objects from memory. Each neuron in this neural network comprises a Hodgkin–Huxley biophysical model with a Mainen modification, the dynamics of which most plausibly reproduce the processes in the neural cells of the brain. In the research on individual neurons, a constant external current with varying amplitudes was applied. The study determined the average periods of self-oscillations and identified parameter values for the Andronov–Hopf bifurcation. In particular, a stable limit cycle is created and destroyed depending on different scenarios when the bias current is increased or decreased. A stable limit cycle arises via the Andronov–Hopf bifurcation and is terminated via the saddle-node bifurcation on the cycle. The study identified parameter regions whereby two neurons with excitatory and inhibitory synaptic connections synchronize. The neurons were all in self-oscillating mode and exhibited a stable limit cycle in the phase space. A three-layer neural network was constructed, comprising of a reference neuron, sensory, control, and interneurons. The control and intermediate layer connections are arranged based on the Hebbian learning rule. Areas of excitation for various types of neurons were found. Experiments were conducted using the neural network architecture to distinguish binary information patterns encoded through signal phase. Excitatory and inhibitory synaptic connections were employed to encode the images stored in memory. The dynamic patterns are determined through the in-phase or anti-phase mode of phase locking, i.e., phase synchronization relative to the global rhythm. Before reaching the output layer of neurons, the signals are summed on the interneuron layer with the signal of the reference neuron, resulting in the filtering out of a certain information segment (in-phase or anti-phase). The network accurately recognized the patterns after parameter adjustments were made. Areas of neuron phase synchronization were discovered, enabling control of network activity modes. The confirmation of the viability of phase capture and pattern recognition modes using Hodgkin–Huxley–Mainen neurons was achieved. In total, as a result of the implementation of the spike neural network modeling project: 1) two-parameter diagrams of regions of neuronal excitation in various modes are calculated; 2) the effect of excitatory and inhibitory coupling in synaptic currents is used to represent the input signal; 3) an abstract mathematical algorithm for calculating the Hebbian matrix has been transferred to the implementation of synaptic currents between layers of neurons; 4) the effect of phase clusters is used to represent the output pattern; 5) the neurons’ topology in the visual-cerebral department served as a working model for recognizing graphic images through the Hopfield network on a spike neural network, using the Hebbian rule.

The Sixth International Symposium on Neural Networks (ISNN 2009)

Making Artificial Brains: Components, Topology, and Optimization.

Abstract-Hebbian learning is widely accepted in the fields of psychology, neurology, and neurobiology. It is one of the fundamental premises of neuroscience. The LMS (least mean square) algorithm of Widrow and Hoff is the world's most widely used adaptive algorithm, fundamental in the fields of signal processing, control systems, pattern recognition, and artificial neural networks. These are very different learning paradigms. Hebbian learning is unsupervised. LMS learning is supervised. However, a form of LMS can be constructed to perform unsupervised learning and, as such, LMS can be used in a natural way to implement Hebbian learning. Combining the two paradigms creates a new unsupervised learning algorithm that has practical engineering applications and provides insight into learning in living neural networks. A fundamental question is, how does learning take place in living neural networks? Nature's little secret, the learning algorithm practiced by nature at the neuron and synapse level, may well be the Hebbian-LMS algorithm.