Dynamic neural network control for non-linear systems: optimal neural network structure and stability analysis

In this paper, the position of a pulse width modulation (PWM)-driven pneumatic actuator has been controlled using a dynamic neural network (DNN) and Proportional Integral Derivative (PID) controller. The harmony search algorithm (HSA) has been used to unravel the optimization problem. The DNN controller is optimally designed to control the position of the actuator. As to the performance of the PID controller, it can assist the DNN controller to give better results. Therefore, an optimal hybrid scheme with both DNN and PID controllers based on HSA is suggested. A pneumatic circuit containing a fast-switching valve is used to reduce the complexity of the PWM-driven servo pneumatic system along with its cost price.

DNNC has been successfully applied to nonisothermal CSTR with time varying parameters including activation energy and heat transfer coefficient. Thus, the objectives of the paper are: 1. To study the constraint handling properties of DNNC, 2. To extend SISO-DNNC to MIMO-DNNC, 3. To study the on-line adaptation properties of DNNC. >

Dynamic neural network controller model of PEM fuel cell system

In order to accelerate the learning ability of neural network structure parameters and improve the prediction accuracy of deep learning algorithms, an evolutionary algorithm, based on a prior Gaussian mutation (PGM) operator, is proposed to optimize the structure parameters of a gated recurrent unit (GRU) neural network. In this algorithm, the sensitivity learning process of GRU model parameters into the Gaussian mutation operator, used the variance of the GRU model parameter training results as the Gaussian mutation variance to generate the optimal individual candidate set. Then, the optimal GRU neural network structure is constructed using the evolutionary algorithm of the prior Gaussian mutation operator. Moreover, the PGM-EA-GRU algorithm is applied to the prediction of stock market returns. Experiments show that the prediction model effectively overcomes the GRU neural network, quickly falling into a local optimum and slowly converging. Compared to the RF, SVR, RNN, LSTM, GRU, and EA-GRU benchmark models, the model significantly improves the searchability and prediction accuracy of the optimal network structure parameters. It also validates the effectiveness and the progressive nature of the PGM-EA-GRU model proposed in this paper with stock market return prediction.

An efficient Q-learning paradigm implemented on a fuzzy CMAC network is proposed. The fuzzy CMAC network topological architecture is described. First, the continuous states of the system are partitioned into a number of fuzzy boxes. Second, the proposed fuzzy CMAC evaluates the Q-values of agents in the fired fuzzy boxes and chooses control actions with maximum Q-values. Then a critic generates an external reinforcement signal according to the outcome or the effect of the control at every time-step, which is used later for further improving the estimation of these Q-values. To speed up the convergence of reinforcement learning, the traditional PID controller with several groups of different parameters is adopted so as to collect a number of taught-lessons. These taught-lessons together with the experienced lessons generated automatically, are sequentially replayed and learned, respectively, under the guidance of different reinforcement mechanisms. The hybrid adaptive and learning control system is applied to the control of a pH-neutralization process. Simulation investigations show that the fuzzy connectionist Q-learning control system has more adaptive, higher intelligence, and stronger generalization ability compared to neural network or fuzzy neural network control techniques using supervised learning.

Optimization of neural network structure using genetic algorithm

In this article we developed a method for optimizing the structure of a fuzzy artificial neural networks through genetic algorithms. This genetic algorithm is used by optimizing the number of weight connections in a neural network structure, by the evolution of those structures as individuals in a population. It is found that the optimization of the neural network provides higher confidence accuracy of the suggested solution in a case based diagnostic system. The computational cost of the optimized network also improved considerably high.

In the applications of the magnetostrictive actuators (MA), hysteresis of the MA is particularly significant and causes undesired effect in the control system. In order to reduce hysteresis effect and obtain precision position in actual application, a neural network supervisory control was proposed and three different neural networks were used. Hysteresis would be compensated and the precision control of the MA would be obtained. These neural networks were respectively radial basis function neural network (RBFNN), dynamic recurrent neural network (DRNN) and cerebellar model articulation controller (CMAC). Through comparing these neural networks in tracking performance, tracking speed and parameter number, the control performance is better when using DRNN and CMAC, especially CMAC with 4 parameters.

In this work realization of automatic scientific articles classification according to Universal Decimal Classifier is presented. Efficiency of neural networks technologies application for current task is researched, and optimal neural network structure and parameters are offered.

In order to improve the quality of magnesia flux pellets and meet the production needs of the iron and steel industry, a pellet formation experiment was carried out. The effects of alkalinity R, SiO2 mass fraction, MgO mass fraction on the green pellets’ burst temperature, compressive strength, and falling strength were studied. The results showed that with the increase in alkalinity, the bursting temperature of green pellets decreases, but has no obvious effect on the compressive strength or drop strength; with the increase in SiO2 content, the bursting temperature of green pellets decreases gradually, and the green pellets’ strength also decreases slightly; with the increase in MgO content, the compressive strength of green pellets shows an upward trend, while the falling strength gradually decreases, and the burst temperature of green pellets shows a trend of rising first and then decreasing. The change trend is coupled with the software test data amplification method algorithm, based on the search algorithm of longicorn (MBAS), to expand a small amount of experimental data. Through data analysis and algorithm comparison, an improved generalized regression neural network (CFA-GRNN), based on culture firefly, was proposed to establish an optimization model for green pellet performance prediction. CFA uses the weights in the input layer and hidden layer of GRNN, the weights in the hidden layer and output layer, the threshold of the hidden layer and the threshold of the output layer as codes for optimization. The evolutionary goal is to obtain the most appropriate and optimal neural network structure. The results show that the MBAS algorithm, combined with the experimental research, can expand the effective data to 1000 pieces. Secondly, the green pellets’ burst temperature, compressive strength and falling strength predicted by the improved generalized regression neural network are in good agreement with the real values, and the average relative errors were 1.88%, 3.18% and 3.62%, respectively. The error analysis shows that the improved model algorithm has higher accuracy, meets the classification of pellets, and can be used to guide the production of pellets.

Artificial neural networks (ANNs) have powerful function approximation and pattern classification capabilities, but their performance is greatly affected by structural design and parameter selection. Traditional training methods have drawbacks including long training time, over-fitting, premature convergence, etc. Evolutionary optimization algorithms have provided an effective tool for ANN parameter optimization, but simultaneously optimizing ANN structure and parameters remains a difficult problem. This paper adapts a relatively new evolutionary algorithm, water wave optimization (WWO), for both structure design and parameter selection for ANNs. The algorithm uses a variable-dimensional solution representation, and designs new propagation, refraction, and breaking operators to effectively evolve solutions towards the optimum or near-optima. Computational experiments show that the WWO algorithm exhibits significant performance advantages over other popular evolutionary algorithms including genetic algorithm, particle swarm optimization, and biogeography-based optimization, for ANN structure and parameter optimization.

This research introduces a method of steganalysis by means of neural networks and its structure optimization. The main aim is to explain the approach of revealing a hidden content in jpeg files by feed forward neural network with Levenberg-Marquardt training algorithm. This work is also concerned to description of data mining techniques for structure optimization of used neural network. The results showed almost 100% success of detection.KeywordsArtificial Neural NetworkHide LayerDiscrete Cosine TransformHide NeuronCover ImageThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Adaptive learning control of robot manipulators via incremental hybrid neural network

<p>Deep neural networks have achieved tremendous success in many domains (e.g., computer vision~\cite{Alexnet12,vggnet15,fastrcnn15}, speech recognition~\cite{hinton2012deep,dahl2012context}, natural language processing~\cite{dahl2012context,collobert2011natural}, games~\cite{silver2017mastering,silver2016mastering}), however, there are still many challenges in deep learning comunity such as how to speed up training large deep neural networks, how to compress large nerual networks for mobile/embed device without performance loss, how to automatically design the optimal network structures for a certain task, and how to further design the optimal networks with improved performance and certain model size with reduced computation cost.</p> <p>To speed up training large neural networks, we propose to use multinomial sampling for dropout, i.e., sampling features or neurons according to a multinomial distribution with different probabilities for different features/neurons. To exhibit the optimal dropout probabilities, we analyze the shallow learning with multinomial dropout and establish the risk bound for stochastic optimization. By minimizing a sampling dependent factor in the risk bound, we obtain a distribution-dependent dropout with sampling probabilities dependent on the second order statistics of the data distribution. To tackle the issue of evolving distribution of neurons in deep learning, we propose an efficient adaptive dropout (named evolutional dropout) that computes the sampling probabilities on-the-fly from a mini-batch of examples.</p> <p>To compress large neural network structures, we propose a simple yet powerful method for compressing the size of deep Convolutional Neural Networks (CNNs) based on parameter binarization. The striking difference from most previous work on parameter binarization/quantization lies at different treatments of $1\times 1$ convolutions and $k\times k$ convolutions ($k>1$), where we only binarize $k\times k$ convolutions into binary patterns. By doing this, we show that previous deep CNNs such as GoogLeNet and Inception-type Nets can be compressed dramatically with marginal drop in performance. Second, in light of the different functionalities of $1\times 1$ (data projection/transformation) and $k\times k$ convolutions (pattern extraction), we propose a new block structure codenamed the pattern residual block that adds transformed feature maps generated by $1\times 1$ convolutions to the pattern feature maps generated by $k\times k$ convolutions, based on which we design a small network with $\sim 1$ million parameters. Combining with our parameter binarization, we achieve better performance on ImageNet than using similar sized networks including recently released Google MobileNets.</p> <p>To automatically design neural networks, we study how to design a genetic programming approach for optimizing the structure of a CNN for a given task under limited computational resources yet without imposing strong restrictions on the search space. To reduce the computational costs, we propose two general strategies that are observed to be helpful: (i) aggressively selecting strongest individuals for survival and reproduction, and killing weaker individuals at a very early age; (ii) increasing mutation frequency to encourage diversity and faster evolution. The combined strategy with additional optimization techniques allows us to explore a large search space but with affordable computational costs.</p> <p>To further design the optimal networks with improved performance and certain model size under reduced computation cost, we propose an ecologically inspired genetic approach for neural network structure search , that includes two types of succession: primary and secondary succession as well as accelerated extinction. Specifically, we first use primary succession to rapidly evolve a community of poor initialized neural network structures into a more diverse community, followed by a secondary succession stage for fine-grained searching based on the networks from the primary succession. Accelerated extinction is applied in both stages to reduce computational cost. In addition, we also introduce the gene duplication to further utilize the novel block of layers that appeared in the discovered network structure.</p>

The Ordovician fissure-cavern carbonate reservoirs of Tarim Basin have relatively large production thickness, long completion intervals and serious heterogeneity problems, former studies about physical parameters for fractured intervals of acid fracturing cannot be well presented through mean algorithm method. This paper uses mean square deviation method to acquire the values of the main influencing factors of acid fracturing, then optimizes the physical characteristics of the fractured intervals. The BP neural network simulation has been adopted to get the optimal BP neural network structure model. In Tarim, this study has simulated 20 wells before acid fracturing using the decision software which compiled by the optimum neural network construction. Comparative analysis has been made through application examples, the coincident rate of mean algorithm is 75%, compared with the 90% of mean square deviation method. Therefore, during the target selection for acid fracturing of heterogeneous carbonate reservoirs with long open hole intervals, it’s significant to use mean square deviation method to optimize physical parameters of reservoirs.