Comparison of different backpropagation training algorithms using robust M-estimators performance functions

In order to estimate daily solar radiation, this paper proposes Elman (ENN) and Feed forward backpropagation (FNN) neural networks. The time series data from the location of Kenitra City, Morocco is used to train the created models. Fletcher-Powell Conjugate Gradient (CGF), Scaled Conjugate Gradient (SCG), Resilient Backpropagation (RB), Conjugate gradient with Powell-Beale restarts (CGB), Levenberg-Marquardt (LM), Polak-Ribi´ere Conjugate Gradient (CGP), and One Step Secant (OSS) are also used with both, the ENN and FNN to identify the best and effective training function for each. The models with various training algorithms are tested by using evaluation metrics root mean square error (RMSE), mean square error (MSE) and mean absolute error (MAE). The study showed that the Elman Neural Network had an excellent daily solar radiation prediction for Kenitra city. Good results have been obtained with Levenberg-Marquardt algorithm.

Air is the most vital constituent for the sustenance of life on earth. Air pollution is the major problem we have been facing. It is important to address this issue to lead a healthy life. Forecasting of air quality will contribute to a healthy society. In this paper, artificial neural network (ANN) predictors trained with conjugate gradient descent have been implemented to forecast air quality index (AQI) in a particular area of interest. Several neural network models such as multilayer perceptron (MLP), Elman, radial basis function and NARX were applied. In these neural network models, four major pollutant concentrations including NO2, CO, O3 and PM10 for the year 2014 to 2016 in Delhi (India) were used to train each predictor. It can be concluded that, among all these models, radial basis function exhibited more accuracy in terms of measures of quality with mean absolute error (MAE) = 7.33, mean absolute percent error (MAPE) = 4.05%, correlation coefficient (R) = 0.993, root mean square error (RMSE) = 9.69 and index of agreement (IA) = 0.99.

Forecasting furrow irrigation infiltration using artificial neural networks

Microwave pyrolysis of oil palm fibre (OPF) was conducted to study the effect of microwave power, temperature, and nitrogen (N2) flow rate on the production of hydrogen and biochar. The effect of microwave power ranging from 400 to 900 W, reaction temperatures ranging from 450°C to 700°C, and N2 flow rates ranging 200 to 1,200 cm3 min−1 were investigated. The microwave power, reaction temperature, and N2 flow rates are directly proportional to hydrogen and inversely proportional to biochar yield. The characteristics of the biochar were analysed using CHNOS, scanning electron microscopy, and Brunauer–Emmett–Teller results. Central composite design was used to optimize the number of experiment. The optimized experimental datasets were used as inputs for developing a model that can predict the weight of hydrogen and biochar from microwave pyrolysis of OPF. A multilayer feedforward neural network model was developed by comparing the performance of 11 different types of backpropagation training algorithms. The 11 training algorithms belong to six classes, which are (a) additive momentum, (b) self‐adaptive learning rate, (c) resilient backpropagation, (d) conjugate gradient, (e) quasi‐Newton, and (f) Bayesian regulation (BR). Best performing training algorithm was selected based on the lowest error values computed. The objective of this research is to identify the most suitable training algorithm for this process. Levenberg–Marquardt and BR exhibited very good performance. BR exhibited best performance with lowest error values in predicting weight of hydrogen and biochar. The testing data error values for hydrogen weight is 0.216 root mean square error (RMSE), 0.144 mean absolute error (MAE), and 0.020 mean absolute percentage error (MAPE), and biochar is 0.886 RMSE, 0.489 MAE, and 0.019 MAPE. The optimum number of hidden neuron for this algorithm is 19 with mean square error 0.23. The biochar exhibits porous structure suggesting possible land application.

Nowadays , nonlinear time series and artificial neural networks (ANN) models are used for forecasting in the field of business, agriculture and soon. Recent studies have shown, ANN have been successfully used for forecasting of financial and agriculture data series The classical methods used for time series prediction like Box-Jenkins or ARIMA assumes that there is a linear relationship between inputs and outputs. ANN have more advantages that can approximate to model both linear and nonlinear structures in time series, they are not able to handling both structures equally well. The autoregressive integrated moving average (ARIMA) model and two ANN models namely, Radial basis function neural networks (RBFNN), and Elman recurrent neural networks (ERNN) methods were applied to Hyderabad airport traffic data. The data obtained for 15 years from 2002–2003 to 2016–2017 about domestic and international passenger of International Airport of Hyderabad, India. In this research paper, we compared the performances of ARIMA, RBFNN and ERNN were based on three measures: mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE). The results showed that RBFNN obtained the smallest MAE, MAPE and RMSE in both the modeling and forecasting processes. The performances of the three models ranked in ascending order were: ARIMA, ERNN and the RBFNN model. Keywords: T ime series, forecasting, artificial neural networks, ARIMA models, radial basis function neural networks, and Elman recurrent neural networks Cite this Article R. Ramakrishna, Berhe Aregay, Tewodros Gebregergs. The Comparison in Time Series Forecasting of Air Traffic Data by Autoregressive Integrated Moving Average Model, Radial Basis Function and Elman Recurrent Neural Networks. Research & Reviews: Journal of Statistics . 2018; 7(3): 75–90p.

This paper presents the fusion of artificial intelligence (AI) learning algorithms that are genetic algorithms (GA) and conjugate gradient (CG) methods. Both methods are used to find the optimum weights for the hidden and output layers of feedforward artificial neural network (ANN) model. Each algorithm is presented in separate module and we proposed three different types of Dynamic Connection Strategies (DyConS) for combining both algorithms to improve the recognition performance of spoken Malay speech recognition. Two different GA techniques are used in this research: a mutated GA technique is proposed and compared with the standard GA technique. One hundred experiments with 5000 words are conducted using the proposed DyConS. Owing to previous facts, GA combined with ANN proved to attain certain advantages with sufficient recognition performance. Thus, from the results, it was observed that the performance of mutated GA algorithm when combined with CG is better than standard GA and CG models. Integrating the GA with feed-forward network improved mean square error (MSE) performance and with good connection strategy by this two stage training scheme, the recognition rate is increased up to 99%.

PurposeThis study constructs time series model, artificial neural networks (ANNs) and statistical topologies to examine the volatility and forecast foreign exchange rates. The Mauritian forex market has been utilized as a case study, and daily data for nominal spot rate (during a time period of five years spanning from 2014 to 2018) for EUR/MUR, GBP/MUR, CAD/MUR and AUD/MUR have been applied for the predictions.Design/methodology/approachAutoregressive integrated moving average (ARIMA) and generalized autoregressive conditional heteroskedasticity (GARCH) models are used as a basis for time series modelling for the analysis, along with the non-linear autoregressive network with exogenous inputs (NARX) neural network backpropagation algorithm utilizing different training functions, namely, Levenberg–Marquardt (LM), Bayesian regularization and scaled conjugate gradient (SCG) algorithms. The study also features a hybrid kernel principal component analysis (KPCA) using the support vector regression (SVR) algorithm as an additional statistical tool to conduct financial market forecasting modelling. Mean squared error (MSE) and root mean square error (RMSE) are employed as indicators for the performance of the models.FindingsThe results demonstrated that the GARCH model performed better in terms of volatility clustering and prediction compared to the ARIMA model. On the other hand, the NARX model indicated that LM and Bayesian regularization training algorithms are the most appropriate method of forecasting the different currency exchange rates as the MSE and RMSE seemed to be the lowest error compared to the other training functions. Meanwhile, the results reported that NARX and KPCA–SVR topologies outperformed the linear time series models due to the theory based on the structural risk minimization principle. Finally, the comparison between the NARX model and KPCA–SVR illustrated that the NARX model outperformed the statistical prediction model. Overall, the study deduced that the NARX topology achieves better prediction performance results compared to time series and statistical parameters.Research limitations/implicationsThe foreign exchange market is considered to be instable owing to uncertainties in the economic environment of any country and thus, accurate forecasting of foreign exchange rates is crucial for any foreign exchange activity. The study has an important economic implication as it will help researchers, investors, traders, speculators and financial analysts, users of financial news in banking and financial institutions, money changers, non-banking financial companies and stock exchange institutions in Mauritius to take investment decisions in terms of international portfolios. Moreover, currency rates instability might raise transaction costs and diminish the returns in terms of international trade. Exchange rate volatility raises the need to implement a highly organized risk management measures so as to disclose future trend and movement of the foreign currencies which could act as an essential guidance for foreign exchange participants. By this way, they will be more alert before conducting any forex transactions including hedging, asset pricing or any speculation activity, take corrective actions, thus preventing them from making any potential losses in the future and gain more profit.Originality/valueThis is one of the first studies applying artificial intelligence (AI) while making use of time series modelling, the NARX neural network backpropagation algorithm and hybrid KPCA–SVR to predict forex using multiple currencies in the foreign exchange market in Mauritius.

The recent years have witnessed an increase in the use of newer analytical tools in the field of medicine to assist in diagnostic procedure. Among the new tools, artificial neural networks (ANNs) have received particular attention because of their ability to analyze complex nonlinear data sets. This study suggests that ANNs can be used for the diagnosis of peripheral nerve disorders particularly the carpal tunnel syndrome (CTS) and neuropathy. This paper aims at building a classifier using a feed forward neural network that can distinguish between CTS, neuropathy, and normal controls using a reduced set of measurements or features from nerve conduction study (NCS) data. Three different ANN training algorithms, viz. Levenberg–Marquardt (LM), Conjugate gradient (CGB), and resilient back-propagation (RP) are used to see which algorithm produces better results and has faster training for the application under consideration. The data used were obtained from the Neurology Department, Kannur Medical College, Kerala, India. The obtained resultant confusion matrix indicated only a few misclassifications in all the three cases. The analysis showed that the CGB and RB algorithms provide faster convergence on pattern recognition problems, but the best performance in terms of accuracy is given by the LM algorithm. The accuracy obtained for the LM, CGB, and RB were 98.3%, 97.8%, and 97.2%, respectively. The respective sensitivities were 96.1%, 94.1%, and 94.1%, while the specificities were found to be equal to 99.4%, 98.8%, and 97.5%, respectively. The study aims at showing that ANNs may prove useful in combination with other systems in providing diagnostic and predictive medical opinions. However, it must always be kept in mind that ANNs represent only one form of computer-aided diagnosis, and the clinician's responsibility and overall control of patient care should never be underestimated.

In this study, the artificial neural networks (ANNs) and regression models were used to downscale the simulated outputs of the general circulation models (GCMs). The simulated precipitation for 25.18 º N to 34.51 º N and 45 º E to 60 º E, geopotential height at 850 mb and zonal wind at 200 mb for 12.56° N to 43.25° N and 19.68° E to 61.87° E data sets as the predictors were extracted from ECHAM5 GCM for the period 1960-2005. The observed monthly precipitation data of Abadan, Abadeh, Ahwaz, Bandar Abbas, Bushehr, Shiraz and Fasa stations as the predictand were extracted for the period 1960-2005. The principal components (PCs) of the simulated data sets were extracted and then six PCs were considered as the input file of the ANN and multiple regression models. Also the combinations of the simulated data sets were used as the input file of these models. The periods 1960-2000 and 2001-2005 were considered as the train and test data in the ANN, respectively. The Pearson correlation coefficient and normalized root mean square error results indicated that ANN predicts precipitation more accurate than multiple regression. For the monthly time scale, the geopotential height is the best predictor and for the seasonal time scale (winter) the simulated precipitation is the best predictor in ANN based standardized precipitation principal components.

This thesis deals with recurrent neural networks, a particular class of artificial neural networks which can learn a generative model of input sequences. The input is mapped, through a feedback loop and a non-linear activation function, into a hidden state, which is then projected into the output space, obtaining either a probability distribution or the new input for the next time-step. This work consists mainly of two parts: a theoretical study for helping the understanding of recurrent neural networks framework, which is not yet deeply investigated, and their application to non-linear prediction problems, since recurrent neural networks are really powerful models suitable for solving several practical tasks in different fields. For what concerns the theoretical part, we analyse the weaknesses of state-of-the-art models and tackle them in order to improve the performance of a recurrent neural network. Firstly, we contribute in the understanding of the dynamical properties of a recurrent neural network, highlighting the close relation between the definition of stable limit cycles and the echo state property of an echo state network. We provide sufficient conditions for the convergence of the hidden state to a trajectory, which is uniquely determined by the input signal, independently of the initial states. This may help extend the memory of the network and increase the design options for the network. Moreover, we develop a novel approach to address the main problem in training recurrent neural networks, the so-called vanishing gradient problem. Our new method allows us to train a very simple recurrent neural network, making the gradient not to vanish even after many time-steps. Exploiting the singular value decomposition of the vanishing factors in the gradient and random matrices theory, we find that the singular values have to be confined in a narrow interval and derive conditions about their root mean square value. Then, we also improve the efficiency of the training of a recurrent neural network, defining a new method for speeding up this process. Thanks to a least square regularization, we can initialize the parameters of the network, in order to set them closer to the minimum and running fewer epochs of classical training algorithms. Moreover, it is also possible to completely train the network with our initialization method, running more iterations of it without losing in performance with respect to classical training algorithms. Finally, it is also possible to use it as a real-time learning algorithm, adjusting the parameters to the new data through one iteration of our initialization. In the last part of this thesis, we apply recurrent neural networks to non-linear prediction problems. We consider prediction of numerical sequences, estimating the following input choosing it from a probability distribution. We study an automatic text generation problem, where we need to predict the following character in order to compose words and sentences, and a path prediction of walking mobile users in the central area of a city, as a sequence of crossroads. Then, we analyse the prediction of video frames, discovering a wide range of applications related to the prediction of movements. We study the collision problem of bouncing balls, taking into account only the sequence of video frames without any knowledge about the physical characteristics of the problem, and the distribution over days of mobile user in a city and in a whole region. Finally, we address the state-of-the-art problem of missing data imputation, analysing the incomplete spectrogram of audio signals. We restore audio signals with missing time-frequency data, demonstrating via numerical experiments that a performance improvement can be achieved involving recurrent neural networks.

Speed and accuracy investigations of neural network algorithms for ionospheric modelling at an equatorial region

Spatial scale effects on retrieval accuracy of surface solar radiation using satellite data

Turkey has sufficient solar radiation and radiation period for solar thermal applications since it lies in a sunny belt between 36° and 42° N latitudes. The yearly average solar radiation is 3.6 kWh/m2day, and the total yearly radiation period is ∼2610 h. This study investigates the estimation of solar resources in Turkey using artificial neural networks (ANNs). Scaled conjugate gradient (SCG), Pola-Ribiere conjugate gradient (CGP), and Levenberg-Marquardt (LM) learning algorithms and logistic sigmoid transfer function were used in the network. In order to train the neural network, meteorological data for last three years (2000–2002) from 17 stations (namely cities) spread over Turkey were used as training (11 stations) and testing (6 stations) data. These cities selected can give a general idea about Turkey. Meteorological and geographical data (latitude, longitude, altitude, month, mean sunshine duration, and mean temperature) is used in input layer of network. Solar radiation is in output layer. The maximum mean absolute percentage error was found to be less than 6.7% and R2 values to be about 99.8937% for the testing stations. However, these values were found to be 2.41% and 99.99658% for the training stations. The results indicate that the ANN model seems promising for evaluating solar resource posibilities at the places where there are no monitoring stations in Turkey. The results on the testing stations indicate a relatively good agreement between the observed and the predicted values.

Estimation of Monthly Mean Daily Global Solar Radiation over Bangkok, Thailand Using Artificial Neural Networks

This paper proposes a step-by-step technique for combining basic models that forecast electricity consumption in an artificial neural network by the method of preliminary selection and further hybridization. The reported experiments were conducted using data on hourly electricity consumption at the metallurgical plant AO ArcelorMittal Temirtau in the period from January 1, 2019, to November 30, 2021. The current research is related to the planned introduction of a balancing electricity market. 96 combinations of basic models were compiled, differing in the type of neural network, the set of initial data, the order of lag, the learning algorithm, and the number of neurons in the hidden layer. It has been determined that the NARX-type network is the most optimal architecture to forecast electricity consumption. Based on experimental studies, the number of hidden neurons needed to form a planned daily profile should equal 3 or 4; it is recommended to use the conjugate gradient method as a learning algorithm. When selecting models from three groups, it was revealed that the conjugate gradient method produces better results compared to the Levenberg-Marquardt algorithm. It is determined that the values of the selected RMSE error indicator take values of 23.17, 22.54, and 22.56, respectively, for the first, second, and third data groups. The adaptive hybridization method has been shown to reduce the RMSE error rate to 21.73. However, the weights of the best models with values of 0.327 for the first group of data, and 0.336 for the second and third ones, show that the individual use of a separate combination of models is also applicable. The devised forecasting electricity consumption model can be integrated into an automated electricity metering system