A Pi-Sigma Higher Order Neural Network for Stock Index Forecasting

An Artificial Neural Network (ANN) works by creating connections between different processing elements (artificial neurons). ANNs have been extensively used for Data Mining, which extracts hidden patterns and valuable information from large databases. This paper introduces a new adaptive Higher Order Neural Network (HONN) model and applies it in data mining tasks such as determining breast cancer recurrences and predicting incomes base on census data. An adaptive hyperbolic tangent function is used as the neuron activation function for the new adaptive HONN model. The paper compares the new HONN model against a Multi-Layer Perceptron (MLP) with the sigmoid activation function, an RBF Neural Network with the Gaussian activation function, and a Recurrent Neural Network (RNN) with the sigmoid activation function. Experimental results show that the new adaptive HONN model offers several advantages over conventional ANN models such as better generalisation capabilities as well as abilities in handling missing values in a dataset.

The Sixth International Symposium on Neural Networks (ISNN 2009)

In 1984 Hopfield showed that the time evolution of a symmetric Hopfield neural networks are a motion in state space that seeks out minima in the energy function (i.e., equilibrium point set of Hopfield neural networks). Because high-order Hopfield neural networks have more extensive applications than Hopfield neural networks, and have been discussed on the convergence of the networks. In practice, a neural network is often subject to environmental noise. It is therefore useful and interesting to find out whether the high-order neural network system still approacher some limit set under stochastic perturbation. In this paper, we will give a number of useful bounds for the noise intensity under which the stochastic high-order neural network will approach its limit set. Our result cancels the requirement of symmetry of the connection weight matrix and includes the classic result on Hopfield neural networks, which is a special case of stochastic high-order Hopfield neural networks. In the end, A example is given to verify the effective of our results.

In this chapter, the primary focus is on theoretical and empirical study of functional link neural networks (FLNNs) for classification. We present a hybrid Chebyshev functional link neural network (cFLNN) without hidden layer with evolvable particle swarm optimization (ePSO) for classification. The resulted classifier is then used for assigning proper class label to an unknown sample. The hybrid cFLNN is a type of feed-forward neural networks, which have the ability to transform the non-linear input space into higher dimensional space where linear separability is possible. In particular, the proposed hybrid cFLNN combines the best attribute of evolvable particle swarm optimization (ePSO), back-propagation learning (BP-Learning), and Chebyshev functional link neural networks (CFLNN). We have shown its effectiveness of classifying the unknown pattern using the datasets obtained from UCI repository. The computational results are then compared with other higher order neural networks (HONNs) like functional link neural network with a generic basis functions, Pi-Sigma neural network (PSNN), radial basis function neural network (RBFNN), and ridge polynomial neural network (RPNN).

Business is a diversified field with general areas of specialisation such as accounting, taxation, stock market, and other financial analysis. Artificial Neural Networks (ANN) have been widely used in applications such as bankruptcy prediction, predicting costs, forecasting revenue, forecasting share prices and exchange rates, processing documents and many more. This chapter introduces an Adaptive Higher Order Neural Network (HONN) model and applies the adaptive model in business applications such as simulating and forecasting share prices. This adaptive HONN model offers significant advantages over traditional Standard ANN models such as much reduced network size, faster training, as well as much improved simulation and forecasting errors, due to their ability to better approximate complex, non-smooth, often discontinuous training data sets. The generalisation ability of this HONN model is explored and discussed.

This paper disposes towards an idea to develop a new network model called a Jordan Pi Sigma Neural Network (JPSN) to overcome the drawbacks of ordinary Multilayer Perceptron (MLP) whilst taking the advantages of Pi-Sigma Neural Network (PSNN). JPSN, a network model with a single layer of tuneable weights with a recurrent term added in the network, is trained using the standard backpropagation algorithm. The network was used to learn a set of historical temperature data of Batu Pahat region for five years (2005-2009), obtained from Malaysian Meteorological Department (MMD). JPSN’s ability to predict the future trends of temperature was tested and compared to that of MLP and the standard PSNN. Simulation results proved that JPSN’s forecast comparatively superior to MLP and PSNN models, with the combination of learning rate 0.1, momentum 0.2 and network architecture 4-2-1 andlower prediction error. Thus, revealing a great potential for JPSN as an alternative mechanism to both PSNN and MLP in predicting the temperature measurement for one-step-ahead.KeywordsJordan pi-sigmaNeural networkTemperature forecasting

Hardware-friendly Higher-Order Neural Network Training using Distributed Evolutionary Algorithms

The neural network is one of the best data mining techniques that have been used by researchers in different areas for the past 10 years. Analysis on Indian stock market prediction using deep learning models plays a very important role in today's economy. In this chapter, various deep learning architectures such as multilayer perceptron, recurrent neural networks, long short -term memory, and convolutional neural network help to predict the stock market prediction. There are two different stock market price companies, namely National Stock Exchange and New York Stock Exchange, are used for analyzing the day-wise closing price used for comparing different techniques such as neural network, multilayer perceptron, and so on. Both the NSE and NYSE share their common details, and they are compared with various existing models. When compared with the previous existing models, neural networks obtain higher accuracy, and their experimental result is shown in betterment compared with existing techniques.

Time series forecasting gets much attention due to its impact on many practical applications. Higher-order neural network with recurrent feedback is a powerful technique which used successfully for forecasting. It maintains fast learning and the ability to learn the dynamics of the series over time. In general, the most used recurrent feedback is the network output. However, no much attention has been paid to use network error instead of the network output. For that, in this paper, we propose a novel model which is called Ridge Polynomial Neural Network with Error Feedback (RPNN-EF) that combines the properties of higher order and error feedback recurrent neural network. Three signals have been used in this paper, namely heat wave temperature, IBM common stock closing price and Mackey–Glass equation. Simulation results show that RPNN-EF is significantly faster than other RPNN-based models for one-step ahead forecasting and its forecasting performance is more significant than these models for multi-step ahead forecasting.

In present days, real time navigation depends on Kalman filter to fuse data from Global Positioning System (GPS) and Inertial Navigation System (INS). But there exist drawbacks like long design time, requirement of prior knowledge in fusing data from GPS and INS using Kalman Filter. In order to overcome the drawbacks of Kalman filter, GPS and INS integration are done using Artificial Neural Networks (ANN) like FFNNs. But still there exists certain drawbacks like less accuracy in FFNNs for GPS and INS data integration. Therefore, this paper introduces Higher Order Neural Networks (HONNs) approach for INS and GPS data integration to overcome the drawback. A higher order feed forward neural network architecture with optimum number of nodes is used. In the given architectures, the replacement of summation at each node by multiplication results in more powerful mapping, because of its capability of processing higher-order information from training data. Performance comparison of HONNs with FFNNs shows that the proposed architecture provides satisfactory results in terms of error rate and number of epochs. The HONNs like Multiplicative Neural Network (MNN) module and Sigma-Pi-ANN module and Traditional Feed Forward Neural Networks(FFNNs) like Radial Basis Function Neural Network (RBF) module and Back propagation Neural Network (BPN) module are trained to predict the INS position error and provide accurate position of the moving aircrafts. Sigma-Pi ANN is found to be the best in terms of accuracy, number of epochs and execution time among the two HONNs.

The motivation for this article is to check whether neural network models have remained a superior method for forecasting the EUR/USD exchange rate during the financial crisis of 2007–2009. Alternative neural network architectures (Multi-Layer Perceptron (MLP), Recurrent Neural Network and Higher Order Neural Network (HONN)) are benchmarked against a random walk and a traditional ARMA model, and evaluated in terms of statistical accuracy and through a trading simulation on daily data over the period from January 2000 to February 2009, the period from August 2007 to February 2009, providing the out-of-sample testing period. Transaction costs and a confirmation filter devised to reduce false signals and thus also reduce losses and transaction costs were also taken into consideration. It is shown that the HONN structure gives the overall best results on a simple trading simulation; however, for an advanced trading simulation with a confirmation filter, the MLP outperforms all other models on most performance measures. On the whole, the results show that neural networks are still able to produce forecasts that yield a positive return and are superior to those of linear and more traditional models, with respect to both trading performance and statistical accuracy, even under very volatile market conditions.

The artificial neural network has been proved among the best tools in data mining for classification tasks. The concept of obtaining more accurate classifier with less computational complexity has been gaining importance, because of day by day increase in the data. Several numbers of models have been developed for classification problems. This paper is the depiction of higher order neural networks especially Fibonacci Functional Link Neural Network (FFLNN) for data classification. The coefficients of individual terms in Fibonacci polynomials are smaller than those of individual terms in the classical orthogonal polynomials. Additionally, less number of terms make it a preferable classifier regarding functional expansion. These properties lead this FFLNN to produce more accurate higher order neural network with less computational complexity to tackle the classification problems. Four datasets were collected from KEEL and LIBSVM dataset repositories. Computational results were compared with three benchmarked models including Chebyshev Functional Link Neural Network (CFLNN), Chebyshev Multilayer Perceptron (CMLP) and Multilayer Perceptron Neural Network (MLP). A t-test was applied to check the significance of the proposed classifier based on classification performance. The findings showed that the proposed classifier outperformed all benchmarked models in all evaluation measures.

The data in the real world are complex and varied. When we apply artificial neural networks to simulate the real-world commerce data through simple functions, accuracy could become problematic. To overcome this issue, we could choose Higher Order Neural Network (HONN) models, which can simulate a data set very accurately. This paper has developed a new open-box HONN model with fractional functions, called a Fractional High Frequency Cosine and Sine HONN (FHFCSHONN) model. FHFCSHONN structure and learning algorithm formulas are studied too. This paper also built a new software package called a FHFCSHONN simulator. Based on the commerce data testing results, the FHFCSHONN model's average error is 0.6980%, while the other three HONN model average errors are 4.0160%, 4.2370%, and 4.3556% respectively.

This chapter develops a new nonlinear model, Ultra high frequency Trigonometric Higher Order Neural Networks (UTHONN), for time series data analysis. Results show that UTHONN models are 3 to 12% better than Equilibrium Real Exchange Rates (ERER) model, and 4 – 9% better than other Polynomial Higher Order Neural Network (PHONN) and Trigonometric Higher Order Neural Network (THONN) models. This study also uses UTHONN models to simulate foreign exchange rates and consumer price index with error approaching 0.0000%.

In the multilayer perceptron (MLP), there was a theorem about the maximum number of separable regions (M) given the number of hidden nodes (H) in the input d-dimensional space. We propose a recurrence relation in the high dimensional space and prove the theorem using the expansion of recurrence relation instead of proof by induction. The MLP model has input layer, one hidden layer, and output layer. We use different MLP models on the well log data inversion to test the number of hidden nodes determined by the theorem. The inputs are the first order, second order, and third order features. Higher order neural network (HONN) has the property of more nonlinear mapping. In the experiments, we have 31 simulated well log data. 25 well log data are used for training, and 6 are for testing. The experimental results can support the number of hidden nodes determined by the theorem.