Dynamical Behavior Analysis for Small-World Scale-Free Neural Networks

The Hopfield model is a pioneering neural network model with associative memory retrieval. The analytical solution of the model in mean field limit revealed that memories can be retrieved without any error up to a finite storage capacity of O(N), where N is the system size. Beyond the threshold, they are completely lost. Since the introduction of the Hopfield model, the theory of neural networks has been further developed toward realistic neural networks using analog neurons, spiking neurons, etc. Nevertheless, those advances are based on fully connected networks, which are inconsistent with recent experimental discovery that the number of connections of each neuron seems to be heterogeneous, following a heavy-tailed distribution. Motivated by this observation, we consider the Hopfield model on scale-free networks and obtain a different pattern of associative memory retrieval from that obtained on the fully connected network: the storage capacity becomes tremendously enhanced but with some error in the memory retrieval, which appears as the heterogeneity of the connections is increased. Moreover, the error rates are also obtained on several real neural networks and are indeed similar to that on scale-free model networks.

Scale-free network and small-world network are the most impacting discoveries in the complex networks theories and have already been successfully proved to be highly effective in improving topology structures of wireless sensor networks. However, currently both theories are not jointly applied to have further improvements in the generation of WSN topologies. Therefore, this paper proposes a cluster-structured evolution model of WSNs considering the characteristics of both networks. With introduction of energy sensitivity and maximum limitation of degrees that a cluster head could have, the performance of our model can be ensured. In order to give an overall assessment of lifting effects of shortcuts, four placement schemes of shortcuts are analyzed. The characteristics of small-world network and scale-free network of our model are proved via theoretical derivation and simulations. Besides, we find that, by introducing shortcuts into scale-free wireless sensor network, the performance of the network can be improved concerning energy-saving and invulnerability, and we discover that the schemes constructing shortcuts between cluster heads and the sink node have better promoted effects than the scheme building shortcuts between pairs of cluster heads, and the schemes based on the preferential principle are superior to the schemes based on the random principle.

Both time delay and coupling form are the most important factors in neural networks. The properties of firing rate oscillation and cluster synchronization induced by time delay are studied in random network of different coupling neurons. In previous work, the firing rate oscillation of cortical network was observed at the presence of three factors (time delay, weak sinusoidal signal, and noise). Here, we found that the firing rate oscillation can be induced only by the time delay, and the spike train can be propagated at a certain interval time, which is consistent with the value of delay time. Furthermore, the phenomenon of cluster synchronization occurs in random network, which may originates from network structure, and this connection between the neurons trigger spikes within a time-restricted window, resulting in cluster synchronization between corresponding neurons. These numerical results provide a potential theoretical basis for certain pathological brain rhythms associated with epileptic seizures.

The purpose of this study is to familiarise the reader with the foundations of neural networks. Artificial Neural Networks (ANNs) are algorithm-based systems that are modelled after Biological Neural Networks (BNNs). Neural networks are an effort to use the human brain's information processing skills to address challenging real-world AI issues. The evolution of neural networks and their significance are briefly explored. ANNs and BNNs are contrasted, and their qualities, benefits, and disadvantages are discussed. The drawbacks of the perceptron model and their improvement by the sigmoid neuron and ReLU neuron are briefly discussed. In addition, we give a bird's-eye view of the different Neural Network models. We study neural networks (NNs) and highlight the different learning approaches and algorithms used in Machine Learning and Deep Learning. We also discuss different types of NNs and their applications. A brief introduction to Neuro-Fuzzy and its applications with a comprehensive review of NN technological advances is provided.

Explosive synchronization, an abrupt transition to a collective coherent state, has been the focus of an extensive research since its first observation in scale-free networks with degree-frequency correlations. In this work, we report several scenarios where a first-order transition to synchronization occurs driven by the presence of a dependence between dynamics and network structure. Therefore, different mechanisms are shown to be able to prevent the formation of a giant synchronization cluster for sufficient large values of the coupling constant in both mono and multilayer networks. Using the Kuramoto model as a reference, we show how for an arbitrary network topology and frequency distribution, a very general weighting procedure acting on the weight of the links delays the synchronization transition forming independent synchronization clusters which suddenly merge above a critical threshold of the coupling constant. A completely different scenario in adaptive and multilayer networks is introduced which gives rise to the emergence of an explosive synchronization when a feedback between the dynamics and structure is operating by means of dependence links weighted through the order parameter.

The purpose of this paper is to familiarize readers with the fundamental ideas of neural networks. The algorithmic models known as Artificial Neural Networks (ANNs) are modeled after Biological Neural Networks (BNNs). Neural networks aim to address challenging AI problems by utilizing the brain's information processing power. The paper provides a concise synopsis of the development and significance of neural networks, contrasting ANNs with BNNs and analyzing their traits, benefits, and drawbacks. Discussion is held regarding the shortcomings of the perceptron model as well as the enhancements brought forth by the sigmoid and ReLU neurons. The paper also looks at different learning strategies and algorithms in machine learning and deep learning, and it gives a summary of several neural network models. In addition to an overview of Neuro-Fuzzy systems, many NN kinds and their uses are examined.

The Bayesian Brain: An Evolutionary Approach to Cognition

Water consumption of plants is a key parameter for formulating irrigation system, and the precise prediction play a important role in improving the use efficiency of limited water resources. In this experiment, by using the method of artificial neural network and MATLAB DATA PROCESSING SYSTEM combined with the meteorological data of air temperature, relative air humidity, solar radiation, wind speed, soil water content and dew point temperature as the input variable, the author established the artificial neural network system to forecast the seedling water consumption of P.×euramericana cv."74/76", and through the experiments it has been examined that two neural network system models can be applied in forecasting water consumption of seedlings, and the average relative error of Back Propagation (BP) neural network prediction model was 0.07, the General Regression Neural Network (GRNN) prediction model was 0.05, moreover, the latter had good stability, while that of the former was poor. Therefore, we propose that GRNN model can be used in prediction of seedling water consumption. Furthermore, the maximum relative error of GRNN predication model was 0.106, the minimum relative error was 0.015. The GRNN model is superior to the BP neural network model that the former performs a higher forecasting accuracy with relatively shorter time consumption and faster speed in training.

We study the influence of network topology on retrieval properties of recurrent neural networks, using replica techniques for diluted systems. The theory is presented for a network with an arbitrary degree distribution $p(k)$ and applied to power law distributions $p(k) \sim k^{-\gamma}$, i.e. to neural networks on scale-free graphs. A bifurcation analysis identifies phase boundaries between the paramagnetic phase and either a retrieval phase or a spin glass phase. Using a population dynamics algorithm, the retrieval overlap and spin glass order parameters may be calculated throughout the phase diagram. It is shown that there is an enhancement of the retrieval properties compared with a Poissonian random graph. We compare our findings with simulations.

This article studies free recall, i.e., the reactivation of stored memory items, namely <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">patterns</i> , in any order, of a model of working memory. Our free recall model is based on a biologically plausible modular neural network composed of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$H$</tex-math></inline-formula> modules, namely <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hypercolumns</i> , each of which is a bundle of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$M$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">minicolumns</i> . The coupling weights and constant bias values of the network are determined by a Hebbian plasticity rule. Using techniques from nonlinear stability theory, we show that cluster synchronization is the central mechanism governing free recall of orthogonally encoded patterns. Particularly, we show that free recall's cluster synchronization is the combination of two main mechanisms: simultaneous activities of minicolumns representing an encoded pattern, i.e., within-pattern synchronization, together with time-divided activities of minicolumns representing different patterns. We characterize the coupling and bias value conditions under which cluster synchronization emerges. We also discuss the role of heterogeneous coupling weights and bias values of minicolumns' dynamics in free recall. Specifically, we compare the behaviour of two <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$H \times 2$</tex-math></inline-formula> networks with identical and non-identical coupling weights and bias values. For these two networks, we obtain bounds on couplings and bias values under which both encoded patterns are recalled. Our analysis shows that having non-identical couplings and bias values for different patterns increases the possibility of their free recall. Numerical simulations are given to validate the theoretical analysis.

An experimental study of pattern recognition of partial discharge (PD) in a crosslinked polyethylene (XLPE) cable by using a neural network (NN) system is described. The NN system was a three-layer artificial neural network system with feedforward connections, and its learning method was a backpropagation algorithm incorporating an external teacher signal. Input information for the NN was a combination of the discharge magnitude, the number of pulse counts and the phase angle of applied voltage in which PD is produced. PD measurement was carried out using a PD pulse recorder for a 66 kV XLPE cable with an artificial defect under a 38 kV AC applied voltage. After learning 30 typical input patterns, the NN discriminated unknown patterns with 90% correct responses. The time duration including measuring time required for the NN to discriminate PD signal was approximately 30 s. In a long-term performance test of a 66 kV XLPE cable with an artificial defect, the NN-based alarm processor was able to recognize the presence of PD 1 h before breakdown of the cable, and successfully alerted the operator. >

Reservoir operation using the neural network and fuzzy systems for dam control and operation support

Neural network, a nonalgorithmic, nondigital, intensely parallel and distributive information processing system, is being used more and more everyday. The main interest in neural networks is rooted in the recognition that the human brain processes information in a different manner than conventional digital computers. Computers are extremely fast and precise at executing sequences of instructions that have been formulated for them. A human information processing system is composed of neurons switching at speeds about a million times slower than computer gates. Yet, humans are more efficient than computers at such computationally complex tasks as speech and other pattern-recognition problems. Artificial neural systems, or neural networks, are physical cellular systems that can acquire, store, and use experiential knowledge. The knowledge is in the form of stable states or mapping embedded in networks that can be recalled in response to the presentation of cues. In a typical neural data processing procedure, the database is divided into two separate portions called training and test sets. The training set is used to develop the desired network. In this process (depending on the paradigm that is being used), the desired output in the training set is used to help the network learn by adjusting the weights between its neurons or processing elements. Once the network has learned the information in the training set and has"converged," the test set is applied to the network for verification. It is important to note that although the user has the desired output of the test set, it has not been seen by the network. This ensures the integrity and robustness of the trained network. A handful of articles on the use of neural networks in the petroleum industry has appeared in SPE conferences, proceedings, and publications in the past 2 years. These articles can be divided into two categories: those that use neural networks to analyze formation lithology from well logs and those that use neural networks to pick a reservoir model to be used in conventional welltest interpretation studies. These tasks are usually done by log analysts and reservoir engineers, and their automation using a fault-tolerant process may prove valuable. Neural networks can help engineers and researchers by addressing some fundamental petroleum engineering problems as well as specific ones that conventional computing has been unable to solve. Petroleum engineers may benefit from neural networks on occasions when engineering data for design and interpretations are less than adequate. This is an especially common occurrence in the Appalachian basin, where some fields are quite old. Lack of adequate engineering data may also be encountered because of the high cost of coring, well testing, and so on. Neural networks have shown great potential for generating accurate analysis and results from large amounts of historical data that otherwise would seem not to be useful or relevant in the analysis. An example of such a problem was encountered by a gas company for a gas storage field in Ohio. In the absence of appropriate data, which normally would make engineering design and evaluation of the fracturing jobs virtually impossible, a carefully designed neural network was able to predict the performance of fracturing jobs with great accuracy. A linear plot of the actual fracturing job results (data never seen by the network during training) and network predictions resulted in a correlation coefficient of 0.98, where 1.00 is a perfect match. Neural networks have proved to be valuable pattern-recognition tools. They are capable of finding highly complex patterns within large amounts of data. A relevant example is well log interpretation. It is generally accepted that there is more information embedded in well logs than meets the eye. Determination, prediction, or estimation of formation permeability without actual laboratory measurement of the cores or interruption in production for well test data collection has been a fundamental problem for petroleum engineers. From geophysical well log data, it was possible to predict and/or estimate permeability of a highly heterogeneous formation in West Virginia, as shown in Fig 1. Reference Mohaghegh, S., Arefi, R., and Ameri, S.: "Design and Development of an Artificial Neural Network for Prediction of Formation Permeability,"paper SPE 28237 presented at the 1994 SPE Petroleum Computer Conference, July31-Aug. 3, Dallas.

The aim of this text is to show the central role played by networks in complex system science. A remarkable feature of network studies is to lie at the crossroads of different disciplines, from mathematics (graph theory, combinatorics, probability theory) to physics (statistical physics of networks) to computer science (network generating algorithms, combinatorial optimization) to biological issues (regulatory networks). New paradigms recently appeared, like that of ‘scale-free networks’ providing an alternative to the random graph model introduced long ago by Erdos and Renyi. With the notion of statistical ensemble and methods originally introduced for percolation networks, statistical physics is of high relevance to get a deep account of topological and statistical properties of a network. Then their consequences on the dynamics taking place in the network should be investigated. Impact of network theory is huge in all natural sciences, especially in biology with gene networks, metabolic networks, neural networks or food webs. I illustrate this brief overview with a recent work on the influence of network topology on the dynamics of coupled excitable units, and the insights it provides about network emerging features, robustness of network behaviors, and the notion of static or dynamic motif.

In this paper, we consider cluster synchronization in an uncertain generic neural network. The network is composed of many interactional clusters, including inner couplings in each cluster and outer couplings between different clusters. For the neural network, the dynamics of each cluster can be unknown and different. The coupling functions including uniform and nonuniform couplings between clusters can be unknown, too. By adding some time-varying linear and negative feedback controllers, we can guarantee that every cluster will synchronize at an arbitrary pre-assigned state/orbit of an isolated cluster of the network. Numerical simulations are presented to show the validity of the theoretical results.