Time-series prediction with BEMCA approach: Application to short rainfall series

Fil: Rodriguez Rivero, Cristian. Universidad Nacional de Córdoba. Facultad de Ciencias Exactas, Físicas y Naturales. Departamento de Ingeniería Electrónica; Argentina.

This paper presents a novel framework for the complexity analysis of rainfall, runoff, and runoff coefficient (RC) time series using multiscale entropy (MSE). The MSE analysis of RC time series was used to investigate changes in the complexity of rainfall-runoff processes due to human activities. Firstly, a coarse graining process was applied to a time series. The sample entropy was then computed for each coarse-grained time series, and plotted as a function of the scale factor. The proposed method was tested in a case study of daily rainfall and runoff data for the upstream Wu–Tu watershed. Results show that the entropy measures of rainfall time series are higher than those of runoff time series at all scale factors. The entropy measures of the RC time series are between the entropy measures of the rainfall and runoff time series at various scale factors. Results also show that the entropy values of rainfall, runoff, and RC time series increase as scale factors increase. The changes in the complexity of RC time series indicate the changes of rainfall-runoff relations due to human activities and provide a reference for the selection of rainfall-runoff models that are capable of dealing with great complexity and take into account of obvious self-similarity can be suggested to the modeling of rainfall-runoff processes. Moreover, the robustness of the MSE results were tested to confirm that MSE analysis is consistent and the same results when removing 25% data, making this approach suitable for the complexity analysis of rainfall, runoff, and RC time series.

Nowadays , nonlinear time series and artificial neural networks (ANN) models are used for forecasting in the field of business, agriculture and soon. Recent studies have shown, ANN have been successfully used for forecasting of financial and agriculture data series The classical methods used for time series prediction like Box-Jenkins or ARIMA assumes that there is a linear relationship between inputs and outputs. ANN have more advantages that can approximate to model both linear and nonlinear structures in time series, they are not able to handling both structures equally well. The autoregressive integrated moving average (ARIMA) model and two ANN models namely, Radial basis function neural networks (RBFNN), and Elman recurrent neural networks (ERNN) methods were applied to Hyderabad airport traffic data. The data obtained for 15 years from 2002–2003 to 2016–2017 about domestic and international passenger of International Airport of Hyderabad, India. In this research paper, we compared the performances of ARIMA, RBFNN and ERNN were based on three measures: mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE). The results showed that RBFNN obtained the smallest MAE, MAPE and RMSE in both the modeling and forecasting processes. The performances of the three models ranked in ascending order were: ARIMA, ERNN and the RBFNN model. Keywords: T ime series, forecasting, artificial neural networks, ARIMA models, radial basis function neural networks, and Elman recurrent neural networks Cite this Article R. Ramakrishna, Berhe Aregay, Tewodros Gebregergs. The Comparison in Time Series Forecasting of Air Traffic Data by Autoregressive Integrated Moving Average Model, Radial Basis Function and Elman Recurrent Neural Networks. Research & Reviews: Journal of Statistics . 2018; 7(3): 75–90p.

Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction

Time series analysis and prediction is an important means of dynamic system modeling. A new method of time series prediction based on support vector regression (SVR) is introduced to resolve the problem of non-linear system modeling. For the purpose of reducing calculation complexity, smooth method is presented to improve standard SVR arithmetic, and is utilized to build the combustion state model of flame in the furnace of utility boilers according to the feature parameters of flame image, in order to predict the combustion state of flame. The flame images are gained from the flame image gathering system on-line. The feature parameters of the flame image are extracted, and are used to determined combustion indices which can denote different combustion states of flame. The time series of combustion indices are used for constructing the smooth support vector regression (SSVR) model and predicting the combustion state of flame. The results of experimentation indicate that SSVR has excellent performance on time series prediction. Compared with traditional time series prediction method such as artificial neural network, SSVR has faster convergence speed and higher fitting precision, which effectively extends the application of SVR.

This study compares the one-day-ahead stream flow forecasting performance of multiple-layer artificial neurons and a neuro-wavelet hybrid system at two sites. Morlet power spectra are used to identify the period-scale structure of the available rainfall and runoff time series. The time series are wavelet decomposed into three sub-series depicting the rainfall-runoff processes: short, intermediate, and long wavelet periods. Then, multiple-layer artificial neurons are trained for each wavelet sub-series. Results show that the short wavelet periods are responsible for most of the final neuro-wavelet hybrid forecasting error. Short period fluctuations are thus the key to any further improvements in artificial neural network (ANN) rainfall-runoff forecasting models. The final performance of the neuro-wavelet hybrid forecasting system and of the classic forecasting multiple-layer artificial neuron system is very similar. The slight advantage in performance of the neuro-wavelet system may be attributed to a better usage of the evapotranspiration time series. Key words: surface-water hydrology, rainfall-runoff, artificial neural networks, wavelet decomposition.

Drought is one of the most important natural hazards in Iran. It is especially more prevalent in arid and hyper arid regions where there are serious limitations in regard to providing sufficient water resources. On the other hand, drought modeling and particularly its prediction can play important role in water resources management under conditions of lack of sufficient water resources. Therefore, in this study, drought prediction in a hyper arid location of Iran (Ardakan region) has been surveyed based on the abilities of artificial neural. Standardized Precipitation Index (SPI) in different time scales (3, 6, 9, 12, and 24 monthly time series) computed based on the data gathered from four rain gauge stations. After evaluation and testing of different artificial neural networks (ANN) structures, gradient descent back propagation (traingd) network showed higher abilities than others. Then, the predictions of SPI time series with different monthly lag times (1:12 months) were tested. Generally, drought prediction by ANNs in the Ardakan region has shown considerable results with the correlation coefficient (R) more than 0.79 and in the most cases and it rises more than 0.90, which indicates the ANN’s ability of drought prediction.

This research introduces a hybrid model for forecasting river flood events with an example of the Mohawk River in New York. Time series analysis and artificial neural networks are combined for the explanation and forecasting of the daily water discharge using hydrogeological and climatic variables. A low pass filter (Kolmogorov–Zurbenko filter) is applied for the decomposition of the time series into different components (long, seasonal, and short-term components). For the prediction of the water discharge time series, each component has been described by applying the multiple linear regression models (MLR), and the artificial neural network (ANN) model. The MLR retains the advantage of the physical interpretation of the water discharge time series. We prove that time series decomposition is essential before the application of any model. Also, decomposition shows that the Mohawk River is affected by multiple time scale components that contribute to the hydrologic cycle of the included watersheds. Comparison of the models proves that the application of the ANN on the decomposed time series improves the accuracy of forecasting flood events. The hybrid model which consists of time series decomposition and artificial neural network leads to a forecasting up to 96% of the explanation for the water discharge time series.

The current research reports a univariate analysis of 2 important climatological parameters surface air temperature and rainfall over North East India over annual scale characterized by various degrees of non-linearity. An Autocorrelation study reveals that although the surface air temperature is characterized by an approximate sinusoidal pattern, the rainfall has no apparent pattern. As both the time series have continuous random variables associated with it we need a discretization to make them suitable for study in time domain. First, we test for Markovian behavior against serial independence. By appropriate discretization of both the time series the transition probabilities are computed to test for Two State Markov Chain model with different orders. A chi square test reveals that although the surface air temperature time series follows the Markov Chain model of 1st Order, the annual rainfall is serially independent. The steady state transition probabilities are computed for surface air temperature and subsequently autoregressive models upto order 4 with the help of Yule-Walker Equations. It was further observed that the 2nd order model satisfied the stationarity conditions. Subsequently to have a general overview of the stationarity, Unit Root Test is carried out for both the time series. For surface air temperature a root was found to exist inside the circumference of the unit circle while for rainfall time series finds a root of the characteristic equation outside unit circle. This indicates stationarity of the surface air temperature time series for its 1st order differences and non-stationarity of rainfall time series. Based on this, Autoregressive Integrated Moving Average (ARIMA(p,q,d)) model is fitted to the rainfall time series for various orders of Autoregression and differencing. Subsequently the performance of ARIMA has been compared to univariate artificial neural network model with 4 successive realizations as predictor and the 5th one as predictand. In order to test the relative contributions to these meteorological phenomenon, maximization of Shannon entropy is used.

A hybrid approach, incorporating concepts of nonlinear dynamics in artificial neural networks, is proposed to study complex dynamic system for more accurate predictions of time series. In this approach important features of nonlinear dynamics are used in the framework of neural networks to construct a model for the underlying dynamics of time series. During pre-analysis phase the series has been characterized in terms of auto-correlation, power spectrum, average mutual information, number of false nearest neighbors, Lyapunov exponents, DVS (deterministic Vs stochastic) plot, and surrogate data sets to find if the series is nonlinear, deterministic or chaotic. The series is then projected in the embedding space by constructing embedding vectors using the method of delays. We examine the dynamics of the system in the embedding space, and note that the time development now follows vector to vector mapping. The artificial neural network has been used to obtain this mapping function. It was observed that time series prediction is better if vector at time T is mapped with more than one vectors immediately preceding it. We illustrate our method by considering a time series generated from the Lorenz equations. The model thus developed gave excellent quality of fit both for the training and test sets. Using the model, the multistep predictions for future 50 values have been made and are found to be very accurate.

Artificial neural networks have been used for time series modeling and forecasting in many domains. However, they are often limited in their handling of nonlinear and chaotic data. More recently, reservoir-based recurrent neural net systems, most notably echo state networks (ESN), have made substantial improvements for time series modeling. Their shallow nature lends themselves to an efficient training method, but has limitations on nonstationary, nonlinear chaotic time series, particularly large, multidimensional time series. In this paper, we propose a novel approach for forecasting time series data based on an additive decomposition (AD) applied to the time series as a preprocessor to a deep echo state network. We compare the performance of our method, AD-DeepESN, on popular neural net architectures used for time series prediction. Stationary and nonstationary data sets are used to evaluate the performance of the methods. Our results are compelling, demonstrating that AD-DeepESN has superior performance, particularly on the most challenging time series that exhibit non-stationarity and chaotic behavior compared to existing methods.

This study aimed at characterization of rainfall dynamics in a hot arid region of Gujarat, India by employing time-series modeling techniques and sustainability approach. Five characteristics, i.e., normality, stationarity, homogeneity, presence/absence of trend, and persistence of 34-year (1980–2013) period annual rainfall time series of ten stations were identified/detected by applying multiple parametric and non-parametric statistical tests. Furthermore, the study involves novelty of proposing sustainability concept for evaluating rainfall time series and demonstrated the concept, for the first time, by identifying the most sustainable rainfall series following reliability (R y), resilience (R e), and vulnerability (V y) approach. Box–whisker plots, normal probability plots, and histograms indicated that the annual rainfall of Mandvi and Dayapar stations is relatively more positively skewed and non-normal compared with that of other stations, which is due to the presence of severe outlier and extreme. Results of Shapiro–Wilk test and Lilliefors test revealed that annual rainfall series of all stations significantly deviated from normal distribution. Two parametric t tests and the non-parametric Mann–Whitney test indicated significant non-stationarity in annual rainfall of Rapar station, where the rainfall was also found to be non-homogeneous based on the results of four parametric homogeneity tests. Four trend tests indicated significantly increasing rainfall trends at Rapar and Gandhidham stations. The autocorrelation analysis suggested the presence of persistence of statistically significant nature in rainfall series of Bhachau (3-year time lag), Mundra (1- and 9-year time lag), Nakhatrana (9-year time lag), and Rapar (3- and 4-year time lag). Results of sustainability approach indicated that annual rainfall of Mundra and Naliya stations (R y = 0.50 and 0.44; R e = 0.47 and 0.47; V y = 0.49 and 0.46, respectively) are the most sustainable and dependable compared with that of other stations. The highest values of sustainability index at Mundra (0.120) and Naliya (0.112) stations confirmed the earlier findings of R y–R e–V y approach. In general, annual rainfall of the study area is less reliable, less resilient, and moderately vulnerable, which emphasizes the need of developing suitable strategies for managing water resources of the area on sustainable basis. Finally, it is recommended that multiple statistical tests (at least two) should be used in time-series modeling for making reliable decisions. Moreover, methodology and findings of the sustainability concept in rainfall time series can easily be adopted in other arid regions of the world.

This thesis evaluates the utility of Artificial Neural Networks (ANNs) applied to financial market and macroeconomic forecasting. In application, ANNs are evaluated in comparison to traditional forecasting models to evaluate if their nonlinear and adaptive properties yield superior forecasting performance in terms of robustness and accuracy. Furthermore, as ANNs are data-driven models, an emphasis is placed on the data collection stage by compiling extensive candidate input variable pools, a task frequently underperformed by prior research. In evaluating their performance, ANNs are applied to the domains of: exchange rate forecasting, volatility forecasting, and macroeconomic forecasting. Regarding exchange rate forecasting, ANNs are applied to forecast the daily logarithmic returns of the EUR/USD over a short-term forecast horizon of one period. Initially, the analytic method of Technical Analysis (TA) and its sub-section of technical indicators are utilized to compile an extensive candidate input variable pool featuring standard and advanced technical indicators measuring all technical aspects of the EUR/USD time series. The candidate input variable pool is then subjected to a two-stage Input Variable Selection (IVS) process, producing an informative subset of technical indicators to serve as inputs to the ANNs. A collection of ANNs is then trained and tested on the EUR/USD time series data with their performance evaluated over a 5-year sample period (2012 to 2016), reserving the last two years for out of sample testing. A Moving Average Convergence Divergence (MACD) model serves as a benchmark with the in-sample and out-of-sample empirical results demonstrating the MACD is a superior forecasting model across most forecast evaluation metrics. For volatility forecasting, ANNs are applied to forecast the volatility of the Nikkei 225 Index over a short-term forecast horizon of one period. Initially, an extensive candidate input variable pool is compiled consisting of implied volatility models and historical volatility models. The candidate input variable pool is then subjected to a two-stage IVS process. A collection of ANNs is then trained and tested on the Nikkei 225 Index time series data with their performance evaluated over a 4-year sample period (2014 to 2017), reserving the last year for out-of-sample testing. A GARCH (1,1) model serves as a benchmark with the out-of-sample empirical results finding the GARCH (1,1) model to be the superior volatility forecasting model. The research concludes with ANNs applied to macroeconomic forecasting, where ANNs are applied to forecast the monthly per cent-change in U.S. civilian unemployment and the quarterly per cent-change in U.S. Gross Domestic Product (GDP). For both studies, an extensive candidate input variable pool is compiled using relevant macroeconomic indicator data sourced from the Federal Bank of St Louis. The candidate input variable pools are then subjected to a two-stage IVS process. A collection of ANNs is trained and tested on the U.S. unemployment time series data (UNEMPLOY) and U.S. GDP time series data. The sample periods are (1972 to 2017) and (1960 to 2016) respectively, reserving the last 20% of data for out of sample testing. In both studies, the performance of the ANNs is benchmarked against a Support Vector Regression (SVR) model and a Naïve forecast. In both studies, the ANNs outperform the SVR benchmark model. The empirical results demonstrate that ANNs are superior forecasting models in the domain of macroeconomic forecasting, with the Modular Neural Network performing notably well. However, the empirical results question the utility of ANNs in the domains of exchange rate forecasting and volatility forecasting. A MACD model outperforms ANNs in exchange rate forecasting both in-sample and out-of-sample, and a GARCH (1,1) model outperforms ANNs in volatility forecasting.

Prediction of the time series from short data samples was considered. An expert-statistical approach based on specially designed methods for joint use of the subjective and objective data was proposed for handling this problem. Its solution relies on the scheme of expert-statistical processing called the analog method.

Surrogate models have been used to replace computationally expensive analysis models in engineering design problems. However, time-dependent variables and historical data are usually ignored in the surrogate modeling process. For instance, in a dam network design, using hydraulic simulations to estimate the water flow is computationally expensive, and the data is in the form of time series. So, we need time-dependent surrogate models to replace these simulations and manage this computational complexity. In this paper, we describe surrogate models to predict the amount of water flow into a reservoir. The challenge is that the flow is a time-dependent variable, and we need to incorporate time series into surrogate models. Thus, there are three contributions: (1) using surrogate modeling to predict flow for dam network design, (2) incorporating time series analysis in surrogate models for water network design, (3) using an ensemble of surrogates to increase the accuracy of prediction. We also demonstrate how to integrate surrogate models and machine learning with time series analysis for more accurate and faster prediction. Due to the availability of data, we use the Buffalo Reservoir in the Red River Basin as an example. Based on the time series data for flow, evaporation, precipitation, and maximum and minimum temperature, three surrogate models are used to examine the impact of integrating time series into surrogate models. These are multivariate autoregressive integrated moving average (MARIMA), a classic time series analysis method; artificial neural network (ANN), and random forest (RF) methods, two machine learning surrogate models. We use seven different time lags as features within an RF model, as an ensemble of surrogate models, and predict the flow for seven-time steps ahead. We successfully incorporate the time series data and particularly the concept of the time lag within surrogate models. We show that RF as the ensemble of surrogates provides more accurate predictions than the other two surrogate models. Although this method has been demonstrated for the Red River Basin, it could also be applied to designing anything in which time-dependent flow is an issue, for example, in biomedical applications, the management of manufacturing processes and product sales as well as any products in which fluid flow is an issue.