Dynamical learning of neural networks based on chaotic dynamics

This paper analyzes on the chaos characteristics of the chaotic neural networks and presents the convergence condition. Although the transient chaos of neural network sould be beneficial to overcome the local minimum problem and speed up the learning, the permanent chaotic response gives adverse effect on optimization problems and makes neural network unstable in general. This paper investigates the dynamic characteristics of the chaotic neural networks with the chaotic dynamic neuron, and presents the convergence condition for stabilizing the chaotic neural networks.

We review our model of chaotic neural networks with chaotic dynamics and apply it to associative memory. We demonstrate that dynamical assocative memory can be realized with the chaotic neural network.

Chaotic behaviors are often shown in the biological brains. They are related strongly to the memory storage and learning in the chaotic neural networks. The incremental learning is a method to compose an associative memory using a chaotic neural network and provides larger capacity than the Hebbian rule in compensation for amount of computation. In the former works, patterns were generated randomly to have plus 1 in half of elements and minus 1 in the others. When finely-tuned parameters were used, the network learned these pattern features, well. But, this result could be taken as an over-learning. Then, we proposed pattern generating methods to avoid over-learning and tested the patterns, in which the ratio of plus 1 and minus 1 is different from 1 to 1. In this paper, our simulations investigate the capacity of the usual chaotic neural network and that of the simplified chaotic neural network with these patterns to ensure no over-learning.

We apply a dimensional analysis to a neural network model with chaotic dynamics, or a mathematical model of chaotic neural networks. First, Lyapunov spectra are examined to confirm existence of chaotic network behavior and to calculate the Lyapunov dimension. Then, the correlation dimension is estimated in the following three cases: (1) reconstruction from single variable time series, (2) observation in state subspaces and (3) observation of the macroscopic average activity. The results show good correlation dimension obtained by observation in state subspaces and the characteristics of the correlation dimension estimated by macroscopic observation.

Design and investigation of a chaotic neural network architecture for cryptographic applications

A multilayer feedforward neural network with a modification to the standard backpropagation training of neural nets is considered. Information on the gradients of target outputs with respect to network inputs is used in this modification. This modified neural network is then applied for approximating design requirements of aerospace composite components, such as an aircraft engine guide vane and a satellite reflector assembly. This neural network functional approximator is then used with an optimization algorithm for determining the optimal parameters for component design requirements. The modified neural network provides for faster convergence of learning process and requires a smaller number of hidden layer nodes for convergence to similar training error levels. HE application of neural networks as universal approximators has been the focus of tremendous amount of activity in the past few years.16 Neural networks have been used for approximating and modeling both memoryless systems and dynamical systems with memory. This paper does not go into the various neural net architectures and training schemes, and the interested reader could refer to standard texts on this subject matter.1'2 This paper considers static neural networks (systems without memory) formed by interconnections without loops, called feedforward networks.2 In addition, this work considers a backpropagation learning scheme with adaptation of learning rate. There have been many additions to, or adaptations of, the basic backpropagation scheme, such as adaptive learning rate, learning with momentum, and the like,1'2 for improving convergence of the learning scheme, and these are not addressed in this paper. This paper explores a modification of the backpropagation learning scheme using information on the gradient of the target outputs with respect to the inputs of the network. The performance of the modified learning scheme is compared with that of the standard backpropagation learning scheme.2 A feedforward network with a single hidden layer trained with this modified learning scheme is applied in structural optimization problems. Figure 1 shows a feedforward net with a single hidden layer. The restriction to a single hidden layer is not a limitation of the modified learning scheme, and the new training scheme can be used on networks with multiple hidden layers also. The use of neural networks for structural response approximation has been reported by several authors.46 The primary reason for investigating such properties is to assess the applicability of trained neural networks for quick and accurate approximation of response functions during structural design optimization. Structural optimization, utilizing either gradient-based numerical search or genetic search algorithms, requires repeated function (response) evaluations during the search process. A single finite element analysis of large-scale structures requires a significant amount of computational time and resource and hence the need for robust approxima

In this paper, anti-synchronization problem of two identical chaotic neural networks with time-varying delays is proposed. By using time-delay feedback control technique, mean value theorem and the Leibniz-Newton formula, and by constructing appropriately Lyapunov-Krasovskii functional, sufficient condition is proposed to guarantee the asymptotically anti-synchronization of two identical chaotic neural networks. This condition, which is expressed in terms of linear matrix inequality, rely on the connection matrix in the drive and response networks as well as the suitable designed feedback gains in the response network. Finally, the anti-synchronization of two chaotic cellular neural network and Hopfield neural network with time-varying delays are considered to illustrate the effectiveness of the proposed control scheme, in which, when compared with the nonlinear feedback control method, the proposed method shows superior performance. Key words: Lyapunov-Krasovskii functional, chaotic neural networks, anti-synchronization, time-varying delay, linear matrix inequality.

Chaotic neural networks have been proved to be strong tools to solve the optimization problems. In order to escape the local minima, a new function is introduced into the self-feedback item of the transiently chaotic neural network, this paper proposes chaotic neural network with nonlinear function self-feedback by improving the self-feedback item of Chen and Aihara's network making that the network has new character which is different from linear self-feedback network. The figures of the reversed bifurcation and the maximal Lyapunov exponents of single neural unit were given, We constructed the energy function of chaotic neural network, and analyzed the sufficient condition for the networks to reach asymptotical stability and set the parameters of the networks for solving traveling salesman problem (TSP). The simulation results in solving 10-city TSP show that the new model is valid in solving optimization problems.

In this paper, the problem on sychronization is investigated for two different Chaotic Neural Network with mixed time-varying Delays. The adaptive synchronization of two different chaotic neural networks with fully unknown parameters and mixed time-varying delays is investigated, and as a result, a well-designed method is given to ensure the synchronization of two chaotic neural networks which are of the different structure and fully unknown parameters. Under the action of the adaptive control law, the synchronization and parametric identification can be done simultaneously. The controller designed is simple and easy to implement. Strict theoretical proofs are given by using the method of Lyapunov function. Finally, numerical simulations show the effectiveness of the method.

Neural networks are able to learn more patterns with the incremental learning than with the correlative learning. The incremental learning is a method to compose an associate memory using a chaotic neural network. In former work, it was found that the capacity of the network increases along with its size, which is the number of the neurons in the network, until some threshold size and that it decreases over that size. The threshold size and the capacity varied between 2 different learning parameters. In this paper, the capacity of the networks was investigated changing the learning parameter. Through the computer simulations, it turned out that the capacity also increases in proportion to the network size in larger sizes and that the capacity of the network with the incremental learning is above 11 times larger than the one with correlative learning.

This paper investigates the performance of current-mode chaotic neural networks that are optimized for travelling salesman problem (TSP). Piece-wise linear (PWL) functions used as the output functions in chaotic neurons are found to ease the design of neural network circuits. Three different PWL output functions are compared by solving 10-city TSP. Simulation results showed excellent performance of the chaotic neural networks with PWL output function. The design parameters to implement the PWL output function suitable for TSP chaotic neural networks are obtained.

Traditional financial prediction models are difficult to cope with complex financial markets, especially cryptocurrency markets. In this paper, a quantum finance-based temporal chaotic neural oscillatory network (TSCNON) prediction model is used for the first time to predict the share price of Bitcoin by combining the quantum price level (QPL) technique with the theory of chaotic neural networks. Based on quantum field signalling (QFS) and Lee oscillator, the overfitting and deadlocking problems of traditional neural networks when dealing with large-scale financial data are solved. The structural design of the TSCNON model and its training algorithm are presented. The application framework of TSCNON in Bitcoin price prediction is demonstrated. Experimental results show that the TSCNON model can greatly reduce the prediction error and improve the prediction accuracy. This paper provides financial market participants with more accurate and reliable prediction tools and promotes the promotion of quantum financial technology in practical applications.

In this paper, a stimulus-response scheme is proposed in chaotic neural networks with synaptic plasticities and the processes of dynamic learning under external stimuli are investigated. Owing to the refractoriness and the time-hysteresis, fixing abilities of patterns of stimuli become much higher than those based on Hopfield neural network and they also have sensitive dependences on the strength of stimulation. These characteristics turn out to be supported with the chaotic activity by examining the relation between the refractoriness and time-hysteresis, and the (maximum) Lyapunov exponent during the engraving of stimuli on the network. The above results indicate the importance of neural chaos when we try to realize the biologically relevant, real-time learning mechanism interactive with the outside, which is difficult for the Hopfield neural network showing poor response behaviors.

This paper presents a new approach for generating broadband noise-like signals by exploiting chaotic dynamics in small recurrent neural networks. A new neural network structure, the chain of chaotic neural networks (CCNN) is introduced in order to obtain a high degree of scrambling for the output signal. Comparing with other chaotic or pseudo-random sequence generators, this one has improved characteristics which make them very attractive for using in various applications where a deterministic but having white-noise like properties signals are needed. Fixed point implementation effects were investigated, simulation results proving that such systems may be efficiently implemented in digital technology and thus offering very well protected ciphering sequences.

This paper proposes algebraic conditions of control for multiple time-delayed chaotic cellular neural networks which can also be applied to chaotic cellular neural networks without time delays. Without many complex restrictions or Lyapunov analytic process, the feedback control which is easy to realize is proposed by utilizing M-matrix theory, the system parameters and the feedback section coefficients. The feasibility and effectiveness of the control are demonstrated via numerical calculations with the help of Matlab. The simulation results also demonstrate the ability of the control to stabilize both the time-delayed chaotic cellular neural networks and some other chaotic cellular neural networks.