Application of NARX Dynamic Neural Network in Quantitative Investment Forecasting System

An increasing number of applications of dynamic neural networks has been developed for digital signal processing (DSP), dynamic neural networks are feedforward neural networks with commonly used scalar synapses replaced by linear filters. This provides feedforward neural networks with the capability of performing dynamic mappings, which depend on past input values, dynamic neural networks are suitable for time series prediction, nonlinear system identification, and signal processing applications. Their most popular types are Finite Impulse Response (FIR) neural networks, which are obtained by replacing synapses with finite impulse response filters. Due to their guaranteed stability characteristic and easy to minimize error surface they have been used with great success in many applications such as signal enhancement, noise cancellation, classification of input patterns, system identification, prediction, and control.. Most of the works on system identification using neural networks are based on multilayer feedforward neural networks with backpropagation learning or more efficient variations of this algorithm, an elegant method for training layered networks. This paper is based on work in a Dynamic System Modeling (DYSMO) and as an application for speed control of DC motor drive.

This paper deals with the adaptive nonlinear identification and trajectory tracking via dynamic multilayer neural network (NN) with different timescales. Two NN identifiers are proposed for nonlinear systems identification via dynamic NNs with different timescales including both fast and slow phenomenon. The first NN identifier uses the output signals from the actual system for the system identification. In the second NN identifier, all the output signals from nonlinear system are replaced with the state variables of the NNs. The online identification algorithms for both NN identifier parameters are proposed using Lyapunov function and singularly perturbed techniques. With the identified NN models, two indirect adaptive NN controllers for the nonlinear systems containing slow and fast dynamic processes are developed. For both developed adaptive NN controllers, the trajectory errors are analyzed and the stability of the systems is proved. Simulation results show that the controller based on the second identifier has better performance than that of the first identifier.

An adaptive identification method of simple dynamic recurr ent neural network (SRNN) for nonlinear dynamic systems is presented in this paper. This method based on the theory that by using the inner-states feed-back of dynamic network to describe the nonlinear kinetic characteris tics of system can reflect the dynamic ch aracteristics more directly, deduces the recursive prediction error (RPE) learning algorithm of SRNNe and improves the algorithm by studying topological structure on recursion layer without the weight values. The simulation results indicate that this kind of neural network can be used in real-time control, due to its less weight values, simpler learning algorithm, higher identification speed, and higher precision of model. It solves the problems of intricate in training algorithm and slow rate in convergence caused by the complicate topologi cal structure in usual dynami c recurrent neural network. Keywords: non-linear system, system identification, dynamic recurrent neural network, RPE algorithm

One of the difficulties encountered in the application of reinforcement learning methods to real-world problem is the generalization of large-scale or continuous state space. In order to solve the curse of dimensionality problem caused by discretizing continuous state space, a kind of Q-learning method for continuous state space based on a dynamic Elman neural network was proposed in this paper. The inputs and the output of Elman network are the system state-action pair and the corresponding Q-value. That is, Elman network is used to estimate the Q-value of state-action pair on-line. Eligibility trace for connecting weights is introduced by borrowing ideas from the eligibility trace mechanism of state in temporal difference algorithm to improve the learning speed of neural network. Computer simulations on mountain car control illustrate the performance and applicability of the proposed Q-learning scheme.

Several mathematical approaches to studying analytically the dynamics of neural networks rely on mean-field approximations, which are rigorously applicable only to networks of infinite size. However, all existing real biological networks have finite size, and many of them, such as microscopic circuits in invertebrates, are composed only of a few tens of neurons. Thus, it is important to be able to extend to small-size networks our ability to study analytically neural dynamics. Analytical solutions of the dynamics of small-size neural networks have remained elusive for many decades, because the powerful methods of statistical analysis, such as the central limit theorem and the law of large numbers, do not apply to small networks. In this article, we critically review recent progress on the study of the dynamics of small networks composed of binary neurons. In particular, we review the mathematical techniques we developed for studying the bifurcations of the network dynamics, the dualism between neural activity and membrane potentials, cross-neuron correlations, and pattern storage in stochastic networks. Then, we compare our results with existing mathematical techniques for studying networks composed of a finite number of neurons. Finally, we highlight key challenges that remain open, future directions for further progress, and possible implications of our results for neuroscience.

The main focus of this paper is to introduce a new supervised learning algorithm for spiking neural networks. The learning algorithm minimizes the overall differences between spike times of target and test spike trains by utilizing a new quantitative similarity measure which has been defined in this work. The actual membrane potential of a post-synaptic neuron is adjusted at the time of spikes based on what has been measured from similarity measure in order to generate the desired membrane potential. Finally, by utilizing gradient descent algorithm, the parameters of the spiking neural network are tuned to generate the desired output membrane potential. The proposed algorithm was applied to tune the facilitation, depression, and synaptic weight constants of the Dynamic Synapses Neural Network - DSNN - for the aim of input-output functional mapping. The simulation results show that the system identification task converges to the global optimum. The rate-to-time coding simulation performs with more than 75 percent accuracy. The performance of both system identification and rate-to-time coding is due to adaptation of short and long term synaptic parameters which cannot be accomplished if only synaptic weight is adapted.

The article is devoted to the problem of optimization of search request ranking algorithms in the digital information retrieval system. The algorithm of functioning of the neural network ranking unit based on Hopfield neural network is built. The ability to generate a ranked list of pages found as a result of the request in the digital information retrieval system can be provided by solving two problems of integer optimization: the problem of assignment of combinatorial sets of criteria for assessing the relevance of web page search and the problem of sorting of numbers—relevance values. The architecture of the neural network model based on the dynamic Hopfield neural network with binary output function designed for combinatorial optimization of the final list of documents found in the digital information retrieval system was synthesized. Promising variants of neural network models with binary output function of neurons for synthesis of the optimal evaluation plan with a combinatorial set of criteria by solving the problem of assignment were built. It has been proven that the built models differ in the rules for determining the coefficients of synaptic connections and external shifts; each of the created rules can be used independently or in different combinations with one another. In the course of analytical research, it was found that the optimization formulation of the problem of sorting of relevance values of search pages is identical to the problem of assignment of combinatorial groups of evaluation criteria provided that the elements of the performance matrix of the latter are defined as linear combinations of relevance values.

The lack of study among the economic forecasting literature that can empirically proves the hypothesis of being more powerfulness of dynamic neural networks in comparison with the static neural networks models for forecasting, is the most important motivation of this study. In this paper, the utilization of NNARX as a nonlinear dynamic neural network model, ANN as a nonlinear static neural network model and ARIMA as a linear model were compared to forecast poultry retail price. As a case study on Iranian poultry retail price, we compare forecast performance of these models for three forecasts (1, 2 and 4 week ahead). Results show that NNARX and ANN models outperform ARIMA model, and also NNARX model outperforms ANN model for all three forecasts.

Online news recommendation aims to continuously select a pool of candidate articles that meet the temporal dynamics of user preferences. Most of the existing methods assume that all user-item interaction history are equally importance for recommendation, which is not alway applied in real-word scenario since the user-item interactions are sometime full of stochasticity and contingency. In addition, previous work on session-based algorithms only considers user sequence behaviors within current session without incorporating users’ historical interests or pointing out users’ main purposes within such session. In this paper, we propose a novel neural network framework, dynamic attention-integrated neural network, to tackle the problems. Specifically, we propose a dynamic neural network to model users’ dynamic interests over time in a unified framework for personalized news recommendations. News article semantic embedding, user interests modelling, session-based public behavior mining and an attention scheme that used to learn the attention score of user and item interaction within sessions are four key factors for online sequences mining and recommendation strategy. Experimental results on three real-world datasets show significant improvements over several baselines and state-of-the-art methods on session-based neural networks.

Paper focuses on the optimization of the advertising and other additional costs for the small business ecommerce web sites. The aim of this paper was to propose a dynamic neural network based algorithm to predict number of clicks on a particular advertising link in three web pages of three different small companies working in the same business segment. The dynamic neural network based algorithm was proposed for forecasting the upcoming values of the advertisement clicks. The best results of 10% average forecasting uncertainty was received for two layer NARX network with three additional inputs: bounce rate, average session time and average page load time. The application of NAR neural network without external inputs increased the forecasting uncertainty to 25% which is similar to received 26% LPC uncertainty.

Biological modeling the undulatory locomotion of C. elegans using dynamic neural network approach

The Sixth International Symposium on Neural Networks (ISNN 2009)

Intelligent Multivariable Modeling of Blast Furnace Molten Iron Quality Based on Dynamic AGA-ANN and PCA

Neural networks are efficient and effective in predicting system states in hydrology. However, most current approaches lack hydrological flow partitioning, do not allow for training on measurements of multiple variables, or lack capability to tightly integrate physically-based components. To address these shortcomings I propose and evaluate an approach referred to below as Dynamical System Neural Network (DSNN). DSNN is a feedforward neural network with an architecture that resembles the organisation in components of the real-world system it represents. In hydrology, the DSNN represents each water flow (e.g. seepage, snow melt) by a collection of input, hidden, and output neural layers, where each input is the state of a hydrological storage (e.g. groundwater storage influencing seepage) or other variable (e.g. air temperature influencing snow melt). These components are interconnected to form a single neural network of the complete dynamical system considered, where all storages and flows are explicitly quantified. If physical understanding of a flow and its parameterization is available, a known formulation can be used as a replacement of a neural network component. The DSNN is applied forward in time, backpropagating gradients over all timesteps. It can be run in spatially lumped or semi-distributed mode. To demonstrate the approach, a DSNN is presented of the Austrian Dorfertal (Kals) Alpine catchment containing snow and subsurface water storages and associated flows including streamflow. The DSNN is trained, validated, and tested on daily streamflow over ~40 years. To explore the capability of the DSNN in estimating the magnitude and dynamics of internal system storages (snow water equivalent, subsurface water storage) and flows (evapotranspiration, sublimation, snowmelt, seepage), the DSNN is first trained and tested with streamflow data generated by a conceptual model. The DSNN turns out to be capable of reproducing - with a satisfactory level of precision - the system states and fluxes calculated by the conceptual model, with decreasing performance when measurement error is added to the artificially generated streamflow data before training. To explore its predictive performance, the DSNN is applied on measured streamflow data for the Dorfertal, comparing multiple DSNN setups that represent all flows as neural network components or only a subset of flows where remaining flows are represented with a standard conceptual model (e.g. linear reservoir). Preliminary results indicate that in predictive performance, in most setups, the DSNN outperforms a standard conceptual model trained on the same streamflow data, with NSE values for testing of 0.74 and 0.71, respectively. This preliminary result indicates DSNN to be a promising approach for blending process-based and neural network based modelling as well as for training (i.e. calibration) of neural network models on measurements of multiple hydrological variables as these are all explicitly represented by the DSNN and can thus be incorporated in the loss function (e.g. streamflow, snow depth, groundwater, evapotranspiration).

The aim of a personalized heating system is to provide a desirable microclimate for each individual when heating is needed. In this paper, we present a method based on machine learning algorithms for generation of predictive models for use in control of personalized heating systems. Data was collected from two individual test subjects in an experiment that consisted of 14 sessions per test subject with each session lasting 4 h. A dynamic recurrent nonlinear autoregressive neural network with exogenous inputs (NARX) was used for developing the models for the prediction of personalized heating settings. The models for subjects A and B were tested with the data that was not used in creating the neural network (unseen data) to evaluate the accuracy of the prediction. Trained NARX showed good performance when tested with the unseen data, with no sign of overfitting. For model A, the optimal network was with 12 hidden neurons with root mean square error equal to 0.043 and Pearson correlation coefficient equal to 0.994. The best result for model B was obtained with a neural network with 16 hidden neurons with root mean square error equal to 0.049 and Pearson correlation coefficient equal to 0.966. In addition to the neural network models, several other machine learning algorithms were tested. Furthermore, the models were on-line tested and the results showed that the test subjects were satisfied with the heating settings that were automatically controlled using the models. Tests with automatic control showed that both test subjects felt comfortable throughout the tests and test subjects expressed their satisfaction with the automatic control.