Pattern recognition by a distributed neural network: An industrial application

The chaotic neural network is studied under the threshold activated coupling, which provides a controlled output patterns. In general the chaotic neural network constructed with chaotic neurons exhibits very rich dynamic behaviors with a nonperiodic associative memory. In the chaotic neural network, it is difficult to distinguish the stored patterns from others, because of the chaotic states of output of the network. In order to apply the nonperiodic associative memory into information search and pattern recognition, etc, it is necessary to control chaos in this chaotic neural network. Chaos in the chaotic neural network is controlled with threshold activated coupling method and the network converges on one of its stored patterns or their reverses which has the smallest Hamming distance with the initial state of the network or provides a controlled neural network with spatio-temporal patterns depending upon the nature of control.

We review our model of chaotic neural networks with chaotic dynamics and apply it to associative memory. We demonstrate that dynamical assocative memory can be realized with the chaotic neural network.

Chaotic dynamics generated in a chaotic neural network model are applied to 2-dimensional (2-D) motion control. The change of position of a moving object in each control time step is determined by a motion function which is calculated from the firing activity of the chaotic neural network. Prototype attractors which correspond to simple motions of the object toward four directions in 2-D space are embedded in the neural network model by designing synaptic connection strengths. Chaotic dynamics introduced by changing system parameters sample intermediate points in the high-dimensional state space between the embedded attractors, resulting in motion in various directions. By means of adaptive switching of the system parameters between a chaotic regime and an attractor regime, the object is able to reach a target in a 2-D maze. In computer experiments, the success rate of this method over many trials not only shows better performance than that of stochastic random pattern generators but also shows that chaotic dynamics can be useful for realizing robust, adaptive and complex control function with simple rules.

In this paper, anti-synchronization problem of two identical chaotic neural networks with time-varying delays is proposed. By using time-delay feedback control technique, mean value theorem and the Leibniz-Newton formula, and by constructing appropriately Lyapunov-Krasovskii functional, sufficient condition is proposed to guarantee the asymptotically anti-synchronization of two identical chaotic neural networks. This condition, which is expressed in terms of linear matrix inequality, rely on the connection matrix in the drive and response networks as well as the suitable designed feedback gains in the response network. Finally, the anti-synchronization of two chaotic cellular neural network and Hopfield neural network with time-varying delays are considered to illustrate the effectiveness of the proposed control scheme, in which, when compared with the nonlinear feedback control method, the proposed method shows superior performance. Key words: Lyapunov-Krasovskii functional, chaotic neural networks, anti-synchronization, time-varying delay, linear matrix inequality.

With the macroeconomy entering a new normal, many new problems are exposed in all walks of life, and the risk of default in the financial sector is also being exposed at an accelerated pace. In the context of big data, internet finance, as an important part of the financial market, also faces many risks in the process of its rapid development. Reasonable, scientific, and effective prediction and prevention of financial default risk have become a key link in the process of risk management practice in the financial industry. Based on the powerful prediction function of the neural network, this paper combined neural network and chaos theory to construct a chaotic RBF neural network. It was applied to financial default risk prediction, which made the prediction accuracy and efficiency higher. The chaotic neural network solves the shortcomings of unstable prediction in the basic neural network and can comprehensively and accurately predict the financial default risk, so as to take measures to prevent risks. The experimental results of this paper show that the accuracy rate of the chaotic RBF neural network reaches 95%, while the accuracy rates of the BP neural network and the RBF neural network are 67% and 78%, respectively. Although the prediction accuracy of these two methods is also high, it is still not as high as the chaotic RBF neural network. Therefore, it is very meaningful to choose the chaotic RBF neural network to predict financial default risk in this paper.

The new approach for combinatorial optimization problems using chaotic dynamics is discussed. We show effectiveness of chaotic neuro dynamics for solving combinatorial optimization problems by applying the chaotic neural network to traveling salesman problems. In this paper, we adopt the chaotic neural network model with two internal states, corresponding to mutual interactions which minimize an energy function and refractoriness which induce chaotic dynamics. We investigate relationships between solving abilities and different model parameters such as decay parameters of two internal states, Lyapunov exponents and first order statistics of firing patterns.

We have expected that dynamic higher functions such as “thinking” emerge through the growth from exploration in the framework of reinforcement learning (RL) using a chaotic Neural Network (NN). In this frame, the chaotic internal dynamics is used for exploration and that eliminates the necessity of giving external exploration noises. A special RL method for this framework has been proposed in which “traces” were introduced. On the other hand, reservoir computing has shown its excellent ability in learning dynamic patterns. Hoerzer et al. showed that the learning can be done by giving rewards and exploration noises instead of explicit teacher signals. In this paper, aiming to introduce the learning ability into our new RL framework, it was shown that the memory-required task in the work of Hoerzer et al. could be learned without giving exploration noises by utilizing the chaotic internal dynamics while the exploration level was adjusted flexibly and autonomously. The task could be learned also using “traces”, but still with problems.

We propose a novel approach for combinatorial optimization using chaotic dynamics. First, we realize the conventional tabu search on a neural network, and we modify it to a chaotic version. Then, our novel method includes both effects of chaotic dynamics and tabu search, which would be effective for combinatorial optimization. We also propose a parameter turning method for effective and fast search of our chaotic neural network. We show that our novel method exhibits better than one of the strongest tabu search, for various quadratic assignment problems, without manual parameter setting.

Information processing and two types of memory in an analog neural network model with time delay that produces chaos similar to the human and animal EEGs are considered. There are two levels of information processing in this neural network: the level of individual neurons and the level of the neural network. Similar to the state of brain, the state of chaotic neural network is defined. It is characterized by two types of memories (memory I and memory II) and correlation structure between the neurons. In normal (unperturbed) state, the neural network generates chaotic patterns of averaged neuronal activities (memory I) and patterns of oscillation amplitudes (memory II). In the presence of external stimulation, the activity patterns change, showing changes in both types of memory. As in experiments on stimulation of the brain, the neural network model shows synchronization of neuronal activities due to stimulus measured by Pearson's correlation coefficient. An increase in neural network asymmetry (increase of the neural network excitability) leads to the phenomenon similar to the epilepsy. Modeling of brain injury, Parkinson's disease, and dementia is performed by removing and weakening interneuron connections. In all cases, the chaotic neural network shows a decrease of the degree of chaos and changes in both types of memory similar to those observed in experiments with healthy human subjects and patients with Parkinson's disease and dementia. © 2005 Wiley Periodicals, Inc. Complexity 11:39–52, 2005

The Sixth International Symposium on Neural Networks (ISNN 2009)

To discover a common and conserved pattern, or motif, from DNA sequences is an important step to analyze DNA sequences because the patterns are acknowledged to reflect biological important information. However, it is difficult to discover unknown motifs from DNA sequences because of its huge number of combination. We have already proposed a new effective method to extract the motifs using a chaotic search, which combines a heuristic algorithm and a chaotic dynamics. To realize the chaotic search, we used a chaotic neural network. The chaotic search exhibits higher performance than conventional methods. Although we have indicated that the refractory effects realized by the chaotic neural network have an essential role, we did not clarify why the refractory effects are important to search optimal solutions. In this paper, we further investigate this issue and reveal the validity of the refractory effects of the chaotic dynamics using surrogate refractory effects. As a result, we discovered that it is important for searching optimal solutions to increase strength of the refractory effects after a firing of neurons.

This study attempts to apply the principle of neural networks and pattern recognition (PR) technologies to real-time recognition by client-server network structure into a web-based recognizing system. In this paper, we recommend a Web-Based PR technology, which is improved recurrent neural network (RNN) from possessing feedback and non-linear activation function with its input, be taken out threshold. The purpose of this article is. to construct a Client-Server network structure for PR system with associative memory. The Server-end is built a databases management system for storage sample patterns. In proceed with training, the user can real-time assign any pattern, which is a record in the Server-end databases. In deal with retrieve task, we propose a novel PR method via databases matching, it can efficient solve spurious states problem from RNN in the WBPR system. On the other hand, taking advantage of Database Matching is to overcome the capacity restrictions on RNN. In order to clarify and corroborate the above Web-Based PR technology, thus a simulation experiment will be presented and their algorithms are also discussed.

Auto-associative neural networks (e.g. the Hopfield model implementing the standard Hebbian prescription) serve as a foundational framework for pattern recognition and associative memory in statistical mechanics. However, their hetero-associative counterparts, though less explored, exhibit even richer computational capabilities. In this work, we examine a straightforward extension of Kosko’s bidirectional associative memory, namely a three-directional associative memory, that is a tripartite neural network equipped with generalized Hebbian weights. Through both analytical approaches (using replica-symmetric statistical mechanics) and computational methods (via Monte Carlo simulations), we derive phase diagrams within the space of control parameters, revealing a region where the network can successfully perform pattern recognition as well as other tasks. In particular, it can achieve pattern disentanglement, namely, when presented with a mixture of patterns, the network can recover the original patterns. Furthermore, the system is capable of retrieving Markovian sequences of patterns and performing generalized frequency modulation.

The dynamics of either associative or hierarchical neural network can boil down to the discovery of fixed point or contraction to the already established fixed point in a discrete dynamical system, and strict mathematical calculations have already proven this point. In other words, the dynamics of all types of neural network can be analyzed and explained by using the fixed point theory in the traditional discrete dynamical system. What is equally important is that the explanation and classification of chaos can also be expressed in terms of its relationship with fixed points. This has provided an important link between chaos and neural network in the traditional discrete dynamical system, namely, the fixed point theory. Based on this idea, this paper proposes a method to combine chaos with neural network using the fixed point theory. With a view to practical application, the paper provides several examples on improving pattern recognition ability by adding chaotic noise in learning machines as well as on improving the ability of optimal solution in the large by creating a new Chaotic Hopfield Neural Network. The approach proposed by this paper is proved user friendly and universally applicable through lab experiments of pattern recognition and solution of the Traveling Salesman Problem on a set of 100 cities.

Construction of blind restoration model for super-resolution image based on chaotic neural network