Time Series Forecasting Using Ridge Polynomial Neural Network with Error Feedback

Motivated by the slow learning properties of multilayer perceptrons which utilize computationally intensive training algorithms and can get trapped in local minima, this work deals with ridge polynomial neural networks (RPNN) and least-square support vector machines (LSSVM) technique. RPNN and LSSVM are combined with the finite element method (FEM), to evaluate the dielectric materials properties. RPNN maintain fast learning properties and powerful mapping capabilities of single layer high order neural networks. LSSVM is a statistical learning method that has good generalization capability and learning performance. Experimental results show that LSSVM can achieve good accuracy and faster speed than those using conventional methods.

In this chapter, the primary focus is on theoretical and empirical study of functional link neural networks (FLNNs) for classification. We present a hybrid Chebyshev functional link neural network (cFLNN) without hidden layer with evolvable particle swarm optimization (ePSO) for classification. The resulted classifier is then used for assigning proper class label to an unknown sample. The hybrid cFLNN is a type of feed-forward neural networks, which have the ability to transform the non-linear input space into higher dimensional space where linear separability is possible. In particular, the proposed hybrid cFLNN combines the best attribute of evolvable particle swarm optimization (ePSO), back-propagation learning (BP-Learning), and Chebyshev functional link neural networks (CFLNN). We have shown its effectiveness of classifying the unknown pattern using the datasets obtained from UCI repository. The computational results are then compared with other higher order neural networks (HONNs) like functional link neural network with a generic basis functions, Pi-Sigma neural network (PSNN), radial basis function neural network (RBFNN), and ridge polynomial neural network (RPNN).

Target Recognition Based on Directional Filter Banks and Higher-Order Neural Networks

In 1984 Hopfield showed that the time evolution of a symmetric Hopfield neural networks are a motion in state space that seeks out minima in the energy function (i.e., equilibrium point set of Hopfield neural networks). Because high-order Hopfield neural networks have more extensive applications than Hopfield neural networks, and have been discussed on the convergence of the networks. In practice, a neural network is often subject to environmental noise. It is therefore useful and interesting to find out whether the high-order neural network system still approacher some limit set under stochastic perturbation. In this paper, we will give a number of useful bounds for the noise intensity under which the stochastic high-order neural network will approach its limit set. Our result cancels the requirement of symmetry of the connection weight matrix and includes the classic result on Hopfield neural networks, which is a special case of stochastic high-order Hopfield neural networks. In the end, A example is given to verify the effective of our results.

Concerned with the slow learning problems of multilayer perceptrons, which utilise computationally intensive training algorithms like the backpropagation learning algorithm that can get trapped in local minima, this paper presents a novel type of higher-order recurrent neural networks, which is called the Dynamic Ridge Polynomial Neural Networks (DRPNN). The aim of the proposed network is to improve the performance of the Ridge Polynomial Neural Network by accommodating recurrent links structure. The network is tested for the prediction of non-linear and non-stationary financial signals. Three exchange rate time-series, which are the exchange rate time series between the British Pound and the Euro, the US dollar and the Euro as well as the exchange rate between the Japanese Yen and the Euro, are used in the simulation process. Simulation results showed that DRPNN generate higher profit returns with fast convergence when used to predict noisy financial time series.

Monotonicity and convergence of asynchronous update gradient method for ridge polynomial neural network

An Artificial Neural Network (ANN) works by creating connections between different processing elements (artificial neurons). ANNs have been extensively used for Data Mining, which extracts hidden patterns and valuable information from large databases. This paper introduces a new adaptive Higher Order Neural Network (HONN) model and applies it in data mining tasks such as determining breast cancer recurrences and predicting incomes base on census data. An adaptive hyperbolic tangent function is used as the neuron activation function for the new adaptive HONN model. The paper compares the new HONN model against a Multi-Layer Perceptron (MLP) with the sigmoid activation function, an RBF Neural Network with the Gaussian activation function, and a Recurrent Neural Network (RNN) with the sigmoid activation function. Experimental results show that the new adaptive HONN model offers several advantages over conventional ANN models such as better generalisation capabilities as well as abilities in handling missing values in a dataset.

In present days, real time navigation depends on Kalman filter to fuse data from Global Positioning System (GPS) and Inertial Navigation System (INS). But there exist drawbacks like long design time, requirement of prior knowledge in fusing data from GPS and INS using Kalman Filter. In order to overcome the drawbacks of Kalman filter, GPS and INS integration are done using Artificial Neural Networks (ANN) like FFNNs. But still there exists certain drawbacks like less accuracy in FFNNs for GPS and INS data integration. Therefore, this paper introduces Higher Order Neural Networks (HONNs) approach for INS and GPS data integration to overcome the drawback. A higher order feed forward neural network architecture with optimum number of nodes is used. In the given architectures, the replacement of summation at each node by multiplication results in more powerful mapping, because of its capability of processing higher-order information from training data. Performance comparison of HONNs with FFNNs shows that the proposed architecture provides satisfactory results in terms of error rate and number of epochs. The HONNs like Multiplicative Neural Network (MNN) module and Sigma-Pi-ANN module and Traditional Feed Forward Neural Networks(FFNNs) like Radial Basis Function Neural Network (RBF) module and Back propagation Neural Network (BPN) module are trained to predict the INS position error and provide accurate position of the moving aircrafts. Sigma-Pi ANN is found to be the best in terms of accuracy, number of epochs and execution time among the two HONNs.

Translational, rotational, and scaling invariant (TRSI) pattern recognition is a third-order problem encountered frequently in real-world applications. But neither traditional image processing/pattern recognition algorithms nor artificial neural networks have yet provided satisfactory solutions for this problem after years of study. Recent research has shown that a higher-order neural network (HONN), of order three with built-in invariances, can effectively achieve TRSI pattern recognition. For an N X N image, the memory needed to store the connections is proportional to N⁶. This huge memory requirement limits the HONNs application to large-scale images. To solve this problem the authors first adapt edge detection and log-spiral mapping algorithms to preprocess the image so that the problem is converted into a second-order one. This reduces the HONN memory requirement to O(N⁴). Second, the authors modified the second-order HONN architecture to further reduce the memory size to O(N²). Synthetic and real images with resolution 256 X 256 have been used for simulation. The training samples are noise free, and the testing samples are rotated, translated, scaled, or noise-corrupted versions of the training patterns. Simulation results show that this system can indeed achieve TRSI pattern classification. In addition, its high robustness to noise and pattern deformation makes it very useful for real-world applications.

In this paper, we propose a novel neural modeling methodology for forecasting daily river discharge that makes use of neural units with higher-order synaptic operations (NU-HSOs). For hydrologic forecasting, conventional rainfall-runoff models based on mechanistic approaches in the literature have shown limitations attributable to their overparameterization and complexity. With the use of neural units with quadratic synaptic operation (NU-QSO) and cubic synaptic operation (NU-CSO), as suggested in this paper, the refined neural modeling methodology can overcome the intricacy and inefficiency of conventional models. In this paper, neural network (NN) models with NU-HSO are compared with conventional NNs with neural units with linear synaptic operation (NU-LSO) for forecasting river discharge. This study was conducted using 1- to 5-day lead time forecasting in the Mahanadi River basin at the Naraj gauging site to evaluate the effectiveness of the higher-order neural networks (HO-NNs). Performance indices for the prediction of daily discharge forecasting indicated that NNs with NU-CSO and NNs with NU-QSO achieved better performance than NNs with NU-LSO even with a lower number of hidden neurons. Thus, this study shows that HO-NNs can be effective in hydrologic forecasting. DOI: 10.1061/(ASCE)HE.1943-5584.0000486. © 2012 American Society of Civil Engineers.

The data in the real world are complex and varied. When we apply artificial neural networks to simulate the real-world commerce data through simple functions, accuracy could become problematic. To overcome this issue, we could choose Higher Order Neural Network (HONN) models, which can simulate a data set very accurately. This paper has developed a new open-box HONN model with fractional functions, called a Fractional High Frequency Cosine and Sine HONN (FHFCSHONN) model. FHFCSHONN structure and learning algorithm formulas are studied too. This paper also built a new software package called a FHFCSHONN simulator. Based on the commerce data testing results, the FHFCSHONN model's average error is 0.6980%, while the other three HONN model average errors are 4.0160%, 4.2370%, and 4.3556% respectively.

This chapter develops a new nonlinear model, Ultra high frequency Trigonometric Higher Order Neural Networks (UTHONN), for time series data analysis. Results show that UTHONN models are 3 to 12% better than Equilibrium Real Exchange Rates (ERER) model, and 4 – 9% better than other Polynomial Higher Order Neural Network (PHONN) and Trigonometric Higher Order Neural Network (THONN) models. This study also uses UTHONN models to simulate foreign exchange rates and consumer price index with error approaching 0.0000%.

This chapter proposes a theoretical framework for parallel implementation of Deep Higher Order Neural Networks (HONNs). First, we develop a new partitioning approach for mapping HONNs to individual computers within a master-slave distributed system (a local area network). This will allow us to use a network of computers (rather than a single computer) to train a HONN to drastically increase its learning speed: all of the computers will be running the HONN simultaneously (parallel implementation). Next, we develop a new learning algorithm so that it can be used for HONN learning in a distributed system environment. Finally, we propose to improve the generalisation ability of the new learning algorithm as used in a distributed system environment. Theoretical analysis of the proposal is thoroughly conducted to verify the soundness of the new approach. Experiments will be performed to test the new algorithm in the future.

Business is a diversified field with general areas of specialisation such as accounting, taxation, stock market, and other financial analysis. Artificial Neural Networks (ANN) have been widely used in applications such as bankruptcy prediction, predicting costs, forecasting revenue, forecasting share prices and exchange rates, processing documents and many more. This chapter introduces an Adaptive Higher Order Neural Network (HONN) model and applies the adaptive model in business applications such as simulating and forecasting share prices. This adaptive HONN model offers significant advantages over traditional Standard ANN models such as much reduced network size, faster training, as well as much improved simulation and forecasting errors, due to their ability to better approximate complex, non-smooth, often discontinuous training data sets. The generalisation ability of this HONN model is explored and discussed.

An improved asynchronous batch gradient method for ridge polynomial neural network