A biologically inspired controller for sound and vibration applications

A biologically inspired control approach for reducing radiated sound power from distributed elastic systems has been experimentally verified for harmonic, narrowband excitation. The control paradigm approximates natural biological systems for initiating movement, hi that a small number of signals are sent from an advanced, centralized controller (analogous to the brain) and are then distributed by local rules and actions to multiple actuators (analogous to muscle fiber). The controller was applied to attenuate radiated sound power from a plate. Experimental investigations of two different local learning rules, the phase variation and optimal methods, were carried out. Performance of the various methods decreased radiated sound power from a clamped plate by up to 16 dB for harmonic off-resonance excitation and up to 22 dB for on-resonance excitation. In general the results have demonstrated that the biological control approach has the potential to control multimodal response in distributed elastic systems using an array of many actuators with a reduced-order main controller. Thus significant reductions in control system computational complexity have been realized by this approach. Nomenclature / = cost function M = number of microphones v = error sensor voltage y = slave actuator complex gains \JL = convergence parameter I. Introduction R ECENT work has demonstrated the potential of active control of distributed elastic systems using multiple, independent actuators and sensors. In work concerned with the control of sound radiation from vibrating panels, the importance of number of channels of control and optimization of the transducer position and shape has been demonstrated.1 However, these investigations were carried out for a fixed frequency, and it is apparent that for good control over a bandwidth of frequencies the control actuators and sensors need to be adaptive in shape. At first sight this problem could be solved using an overall transducer broken up into many individual small elements, each connected by an individual control channel. In this situation the control transducer would effectively reoptimize its configuration for different conditions by adaptively weighting each transducer segment. Meirovitch and Norris2 have demonstrated the advantage of such an approach by considering fully distributed control in reducing control spillover. The disadvantage of this approach is that, for systems with a high modal density, the number of actuators and sensors required becomes extremely large. A high number of control channels has a number of problems mainly associated with memory requirements and computational time in the hardware systems used to implement the control. In addition, collinearity of transducer transfer functions (i.e., the transducer transfer functions are not fully linear independent) causes stability problems in systems with a high number of transducers. A new approach of controlling distributed elastic systems is presented. The approach is inspired by the action of biological natural systems where a small number of main signals are transmitted from the brain to a large area of muscle tissue to activate many independent segments of muscle. The signals then stimulate local action that is governed by, for example, chemical interaction of locally

A biologically inspired control approach for reducing vibrations in distributed elastic systems has been derived and experimentally verified for narrow-band excitation. The control paradigm approximates natural biological systems for initiating movement, in that a low number of signals are sent from an advanced, centralized controller (analogous to the motor cortex of the brain) and are then distributed by local simple rules to multiple control actuators (analogous to muscle fiber). Both theoretical and experimental investigations of three different local rules were carried out including a stability analysis for reducing beam vibrations. In general the results have demonstrated that the biological control approach has the potential to control multimodal response in distributed elastic systems using an array of many actuators with a reduced order main controller. Thus significant reductions in control system computational complexity have been realized by this approach.

A biologically plausible model of myelin plasticity demonstrates how oligodendrocytes can selectively adjust conduction delays in axon bundles to allow robust communication between brain regions through long-range synchronization.

An important problem encountered by both natural and engineered signal processing systems is blind source separation. In many instances of the problem, the sources are bounded by their nature and known to be so, even though the particular bound may not be known. To separate such bounded sources from their mixtures, we propose a new optimization problem, Bounded Similarity Matching (BSM). A principled derivation of an adaptive BSM algorithm leads to a recurrent neural network with a clipping nonlinearity. The network adapts by local learning rules, satisfying an important constraint for both biological plausibility and implementability in neuromorphic hardware.

A fundamental aspect of learning in biological neural networks is the plasticity property which allows them to modify their configurations during their lifetime. Hebbian learning is a biologically plausible mechanism for modeling the plasticity property in artificial neural networks (ANNs), based on the local interactions of neurons. However, the emergence of a coherent global learning behavior from local Hebbian plasticity rules is not very well understood. The goal of this work is to discover interpretable local Hebbian learning rules that can provide autonomous global learning. To achieve this, we use a discrete representation to encode the learning rules in a finite search space. These rules are then used to perform synaptic changes, based on the local interactions of the neurons. We employ genetic algorithms to optimize these rules to allow learning on two separate tasks (a foraging and a prey-predator scenario) in online lifetime learning settings. The resulting evolved rules converged into a set of well-defined interpretable types, that are thoroughly discussed. Notably, the performance of these rules, while adapting the ANNs during the learning tasks, is comparable to that of offline learning methods such as hill climbing.

There have been many attempts to understand complexity and to represent it in terms of computable quantities. To date, however, these attempts have had little success. Although we find complexity in a broad range of scientific domains, precise definitions escape our grasp (Bak, 1996; Morin, 2001; Prigogine & Stengers, 1984). One of the key models in complexity science is the Cellular Automaton (CA), a class of system in which small changes in the initial conditions or in local rules can provoke unpredictable behavior (Wolfram, 1984; Wolfram, 2002; Langton, 1986; 1990). The key issue, here as in other kinds of complex system, is to discover the rules governing the emergence of complex phenomena. If such rules were known we could use them to model and predict the behavior of complex physical and biological systems. Taking it for granted that complex behavior is the result of interactions among multiple components of a larger system; we can ask a number of fundamental questions.

Neuromorphic Systems using Local Learning Rule and Thin-Film Devices

Equilibrium Propagation is a biologically-inspired algorithm that trains convergent recurrent neural networks with a local learning rule. This approach constitutes a major lead to allow learning-capable neuromophic systems and comes with strong theoretical guarantees. Equilibrium propagation operates in two phases, during which the network is let to evolve freely and then “nudged” toward a target; the weights of the network are then updated based solely on the states of the neurons that they connect. The weight updates of Equilibrium Propagation have been shown mathematically to approach those provided by Backpropagation Through Time (BPTT), the mainstream approach to train recurrent neural networks, when nudging is performed with infinitely small strength. In practice, however, the standard implementation of Equilibrium Propagation does not scale to visual tasks harder than MNIST. In this work, we show that a bias in the gradient estimate of equilibrium propagation, inherent in the use of finite nudging, is responsible for this phenomenon and that canceling it allows training deep convolutional neural networks. We show that this bias can be greatly reduced by using symmetric nudging (a positive nudging and a negative one). We also generalize Equilibrium Propagation to the case of cross-entropy loss (by opposition to squared error). As a result of these advances, we are able to achieve a test error of 11.7% on CIFAR-10, which approaches the one achieved by BPTT and provides a major improvement with respect to the standard Equilibrium Propagation that gives 86% test error. We also apply these techniques to train an architecture with unidirectional forward and backward connections, yielding a 13.2% test error. These results highlight equilibrium propagation as a compelling biologically-plausible approach to compute error gradients in deep neuromorphic systems.

Finding informative low-dimensional representations that can be computed efficiently in large datasets is an important problem in data analysis. Recently, contrastive Principal Component Analysis (cPCA) was proposed as a more informative generalization of PCA that takes advantage of contrastive learning. However, the performance of cPCA is sensitive to hyper-parameter choice and there is currently no online algorithm for implementing cPCA. Here, we introduce a modified cPCA method, which we denote cPCA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∗</sup> , that is more interpretable and less sensitive to the choice of hyper-parameter. We derive an online algorithm for cPCA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∗</sup> and show that it maps onto a neural network with local learning rules, so it can potentially be implemented in energy efficient neuromorphic hardware. We evaluate the performance of our online algorithm on real datasets and highlight the differences and similarities with the original formulation.

Sparse coding models of natural scenes can account for several physiological properties of primary visual cortex (V1), including the shapes of simple cell receptive fields (RFs) and the highly kurtotic firing rates of V1 neurons. Current spiking network models of pattern learning and sparse coding require direct inhibitory connections between the excitatory simple cells, in conflict with the physiological distinction between excitatory (glutamatergic) and inhibitory (GABAergic) neurons (Dale's Law). At the same time, the computational role of inhibitory neurons in cortical microcircuit function has yet to be fully explained. Here we show that adding a separate population of inhibitory neurons to a spiking model of V1 provides conformance to Dale's Law, proposes a computational role for at least one class of interneurons, and accounts for certain observed physiological properties in V1. When trained on natural images, this excitatory-inhibitory spiking circuit learns a sparse code with Gabor-like RFs as found in V1 using only local synaptic plasticity rules. The inhibitory neurons enable sparse code formation by suppressing predictable spikes, which actively decorrelates the excitatory population. The model predicts that only a small number of inhibitory cells is required relative to excitatory cells and that excitatory and inhibitory input should be correlated, in agreement with experimental findings in visual cortex. We also introduce a novel local learning rule that measures stimulus-dependent correlations between neurons to support "explaining away" mechanisms in neural coding.

Perceptron-like learning rules are known to require exponentially many correction steps in order to identify boolean threshold functions exactly. We introduce criteria that are weaker than exact identification and investigate whether learning becomes significantly faster if exact identification is replaced by one of these criteria: probably approximately correct (PAC) identification, order identification, and sign identification. PAC identification is based on the learning paradigm introduced by Valiant (1984) and is known to be easier than exact identification. Order identification uses the fact that each threshold function induces an ordering relation on the input variables that can be represented by weights of linear size. Sign identification is based on a property of threshold functions known as unateness and requires only weights of constant size. We show that Perceptron-like learning rules cannot satisfy these criteria when the number of correction steps is to be bounded by a polynomial. We also present an exponential lower bound for order identification with the learning rules introduced by Littlestone (1988). Our results show that efficiency imposes severe restrictions on what can be learned with local learning rules.

We introduce an information maximizing neural network that employs only local learning rules, simple activation functions, and feedback in its functioning. The network consists of an input layer, an output layer that can be overcomplete, and a set of auxiliary layers comprising feed-forward, lateral, and feedback connecwtions. The auxiliary layers implement a novel ”neural multigrid,” and each computes a Fourier mode of a key infomax learning vector. Initially, a partial multigrid computes only low frequency modes of this learning vector, resulting in a spatially correlated topographic map. As higher frequency modes of the learning vector are gradually added, an infomax solution emerges, maximizing the entropy of the output without disrupting the map’s topographic order. When feed-forward and feedback connections to the neural multigrid are passed through a nonlinear activation function, infomax emerges in a phase-independent topographic map. Information rates estimated by Principal Components Analysis (PCA) are comparable to those of standard infomax, indicating the neural multigrid successfully imposes a topographic order on the optimal infomax-derived bases.KeywordsMutual InformationCoarse GridMultigrid MethodFourier ModeNeural ComputationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).

One of the biggest stakes in nanoelectronics today is to meet the needs of Artificial Intelligence by designing hardware neural networks which, by fusing computation and memory, process and learn from data with limited energy. For this purpose, memristive devices are excellent candidates to emulate synapses. A challenge, however, is to map existing learning algorithms onto a chip: for a physical implementation, a learning rule should ideally be tolerant to the typical intrinsic imperfections of such memristive devices, and local. Restricted Boltzmann Machines (RBM), for their local learning rule and inherent tolerance to stochasticity, comply with both of these constraints and constitute a highly attractive algorithm towards achieving memristor-based Deep Learning. On simulation grounds, this work gives insights into designing simple memristive devices programming protocols to train on chip Boltzmann Machines. Among other RBM-based neural networks, we advocate using a Discriminative RBM, with two hardware-oriented adaptations. We propose a pulse width selection scheme based on the sign of two successive weight updates, and show that it removes the constraint to precisely tune the initial programming pulse width as a hyperparameter. We also propose to evaluate the weight update requested by the algorithm across several samples and stochastic realizations. We show that this strategy brings a partial immunity against the most severe memristive device imperfections such as the non-linearity and the stochasticity of the conductance updates, as well as device-to-device variability.

Associative storage and retrieval of binary random patterns in various neural net models with one-step threshold-detection retrieval and local learning rules are the subject of this chapter. For different heteroas-sociation and autoassociation memory tasks specified by the properties of the pattern sets to be stored and upper bounds on the retrieval errors, we compare the performance of various models of finite as well as asymptotically infinite sizes. In infinite models, we consider the case of asymptotically sparse patterns, where the mean activity in a pattern vanishes, and study two asymptotic fidelity requirements: constant error probabilities and vanishing error probabilities. A signal-to-noise ratio analysis is carried out for one retrieval step where the calculations are comparatively straightforward and easy. As performance measures we propose and evaluate information capacities in bits/synapse which also take into account the important property of fault tolerance. For autoassociation we compare one-step and fixed-point retrieval that is analyzed in the literature by methods of statistical mechanics.