An Introduction to ARMA Models

Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir

Three important parametric models for describing the correlation functions and spectra of stationary stochastic processes are the autoregressive (AR), moving average (MA), and autoregressive-moving average (ARMA) models. Quite recently, the MATLAB toolbox “ARMASA” has been made publicly available. This toolbox provides state-of-the-art algorithms to perform automatic identification and selection between the models based on the estimated prediction error. ARMASA works on a single segment of data, whereas in some applications, the data are available as multiple segments. We could process each segment independently and average the estimated autocorrelation functions or spectra afterward. Better performance, however, can be expected when all segments are processed simultaneously, for two reasons. Initially, the bias in the estimated model parameters depends on the number of observations in a segment. Averaging models from segments does not reduce this bias, in contrast to identification from all the data at once. In either case, the variance of the estimated model parameters will decrease. By considering all the data at once this variance reduction can be considered in the order selection stage, decreasing the selection bias. In this paper, we extend the parameter estimation algorithms of the ARMASA toolbox to the case of multiple segments. AR parameter estimation from segments has been considered before, but not for the MA and ARMA models. In addition, in the AR case, model order selection can be done easily when the segments are independent. But, in the application we are primarily interested in, the segments are formed by lines of scanning transmission electron microscopy (STEM) images, which can be correlated with each other. Because the correlation among data segments influences the effective number of independent observations available it becomes necessary to estimate that number. This is not an easy problem in general. For observations from a stochastic process with an isotropic 2-D autocorrelation function we can, however, formulate an accurate estimator in terms of the line autocorrelation function. This situation arises, for example, in STEM images of amorphous specimens. From the AR model identification for segments, the extension to the MA and ARMA models is rather straightforward because MA and ARMA algorithms are based on Durbin's methods, whose first step is the estimation of a long AR model. We compare the segment-based algorithms to conventional ARMASA with simulations of stationary stochastic processes. An illustrative example involving simulated STEM images shows large potential benefits.

Nowadays , nonlinear time series and artificial neural networks (ANN) models are used for forecasting in the field of business, agriculture and soon. Recent studies have shown, ANN have been successfully used for forecasting of financial and agriculture data series The classical methods used for time series prediction like Box-Jenkins or ARIMA assumes that there is a linear relationship between inputs and outputs. ANN have more advantages that can approximate to model both linear and nonlinear structures in time series, they are not able to handling both structures equally well. The autoregressive integrated moving average (ARIMA) model and two ANN models namely, Radial basis function neural networks (RBFNN), and Elman recurrent neural networks (ERNN) methods were applied to Hyderabad airport traffic data. The data obtained for 15 years from 2002–2003 to 2016–2017 about domestic and international passenger of International Airport of Hyderabad, India. In this research paper, we compared the performances of ARIMA, RBFNN and ERNN were based on three measures: mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE). The results showed that RBFNN obtained the smallest MAE, MAPE and RMSE in both the modeling and forecasting processes. The performances of the three models ranked in ascending order were: ARIMA, ERNN and the RBFNN model. Keywords: T ime series, forecasting, artificial neural networks, ARIMA models, radial basis function neural networks, and Elman recurrent neural networks Cite this Article R. Ramakrishna, Berhe Aregay, Tewodros Gebregergs. The Comparison in Time Series Forecasting of Air Traffic Data by Autoregressive Integrated Moving Average Model, Radial Basis Function and Elman Recurrent Neural Networks. Research & Reviews: Journal of Statistics . 2018; 7(3): 75–90p.

Time series modeling and analysis provides means of predicting the future and has been widely used in a variety of fields ranging from seismology for predicting earthquake and volcanic eruption, to finance for risk assessment, and to quantum information processing. The conventional integer order models can only capture short-range dependence; for example, Poisson processes, Markov processes, autoregressive (AR), moving average (MA), autoregressive moving average (ARMA) and autoregressive integrated moving average (ARIMA) processes. In time series analysis, one of the conventional assumptions is that the coupling between values at different time instants decreases rapidly as the time difference or distance increases. However, there are situations where strong coupling between values at different times exhibit properties of long range dependence which cannot be processed by the conventional time series analysis. Typical examples of long range dependence signals include financial time series, underwater noise, electroencephalography (EEG) signal, etc. ARFIMA, a fractional order signal processing technique, is the generalization of the conventional integer order techniques, namely, ARIMA and ARMA methods. Hence, it is capable of capturing both short-range dependence and long-range dependence in signals. Compared to conventional integer order models, the ARFIMA model gives a better fit and result when dealing with the data which possess the long range dependence property. In this paper, we investigate the application of the ARFIMA as well as AR methods to model EEG signals obtained from different brain channels. We analyze the resulting correlations for comparison the benefits of ARFIMA over AR on the EEG data exhibiting the long range dependency property. The results showed that the prediction results have a better performance compared to the conventional ARMA models.

Background/Objective: To provide an efficient predictive technique to foresee future workload as well as to handle the resources efficiently by performing hybrid auto scaling for Cloud applications. Cloud applications might expertise completely different workload at different times, automatic provisioning has to work with efficiency at any point of time. Auto scaling is a feature of cloud computing that potentially scale the resources in line on demand. Considering this expectation, they are generally categorized into Reactive scaling which adds or reduces resources based on a fixed threshold value. The predictive scaling is used provide necessary scaling actions beforehand. Methods/Statistical Analysis: To perform the hybrid auto scaling (reactive plus predictive auto scaling), a time series technique should be used. Auto-regressive Moving Average (ARMA) model, the Exponential Smoothing (ES) model, the Autoregressive model (AR), the Moving Average model (MA) and the Trend- Adjusted Exponential Smoothing (TAES), Auto Regressive Integrated Moving Average (ARIMA) Time-series model, Naïve bayes algorithm, Recurrent Neural Network- Long Short Term Memory (RNN-LSTM), Independent Recurrent Neural Network (IndRNN) are time series techniques used to foresee the future workload. To find the effectiveness of predictive techniques, Mean Absolute Error (MAE), Mean Squared Error (MSE) and Mean Absolute Percentage Error (MAPE) performance metrics are evaluated. Findings: Based on the evaluation, IndRNN gives the minimum error rate. IndRNN is used to predict the future resource requisites in order to ascertain adequate resource are available ahead of time. Application: The predicted result from IndRNN method is integrated on private cloud to autoscale the resources for cloud applications. Keywords: Cloud Applications, Hybrid Autoscaling, Independent Recurrent Neural Network (IndRNN), Private Cloud, Workload

This paper proposes the Discrete Wavelet Transform (DWT), Auto Regressive Integrated Moving Averages (ARIMA) model and Recurrent Neural Network (RNN) based technique for forecasting the computer network traffic. Computer network traffic is sampled on computer networking device connected to the internet. At first, discrete wavelet transform is used to decompose the traffic data into non-linear (approximate) and linear (detailed) components. After that, detailed and approximate components are reconstructed using inverse DWT and predictions are made using Auto Regressive Moving Average (ARIMA) and Recurrent Neural Networks (RNN), respectively. Internet traffic is a time series which can be used to predict the future traffic trends in a computer network. Numerous computer network management tasks depend heavily on the information about the network traffic. This forecasting is very useful for numerous applications, such as congestion control, anomaly detection, and bandwidth allocation. Our method is easy to implement and computationally less expensive which can be easily applied at the data centers, improving the quality of service (QoS) and reducing the cost.

The stationary data tests in time series data are the most common idea to create the model with an autoregressive (AR) model, moving average (MA) model, autoregressive moving average (ARMA) model, and autoregressive integrated moving average (ARIMA) model. In this chapter, we present the tseries package in the R program to investigate stationary data on the time series. This package offers Augmented Dickey-Fuller (ADF), Kwiatkowski-Phillips-Schmidt-Shin (KPSS), and Phillips-Perron (PP) tests. Three tests also provide the statistics test and p-value to decide the acceptance or rejection of the null hypothesis. This chapter also aims to present the calculation of these tests proposes step-by-step and the available coding commands under similar time series data. The performance of these tests considers the probability of type I error and the power of the test. The time series data in this study is simulated from the stationary in term of random walk process and non-stationary in term of MA model. This simulation is also conducted to recommend that the user select the appropriate stationarity tests. The R program employs to simulate and analyzes data on 1,000 replications for several sample sizes. The simulation results showed that the ADF and PP tests could control the best probability of type I error on non-stationary time series data. For the power of the test, ADF and PP tests give the highest power of the test on stationary time series data.

The quarterly electricity consumption curve shows certain regularity due to factors such as temperature and holidays. The ARMA(Autoregressive Moving Average) model has a good fitting ability for such time series, but the quarterly electricity consumption is also affected by economic growth. It is an ascending time series, which is a nonstationary time series. It cannot be directly predicted by ARMA model. Therefore, it is necessary to make finite difference processing to the original time series to stationary time series. The original time series is suitable for ARIMA(Autoregressive Integrated Moving Average) model. Any prediction model can not avoid the generation of residuals. The K-means clustering algorithm is used to divide the residual sequence state. By exploring the relationship between states at different moments, the transition matrix is established to predict the probability distribution of the residual state at the next moment. The predicted state is restored to the residual prediction value, and the prediction of the residual is completed to implement the correction of the original prediction value. That is, by predicting the residual by establishing a Markov model, the predicted value of the ARIMA model is corrected, and the prediction accuracy is improved.

Speech applications in noisy and degraded channel conditions continue to be a challenging problem especially when there is a mismatch between the training and test conditions. In this paper, a robust speech feature extraction scheme is developed based on autoregressive moving average (ARMA) modeling that emphasizes high energy regions of the signal with a data driven modulation filter. The peak preserving ability of two dimensional autoregressive (AR) models is used to emphasize the high energy regions in the spectrotemporal domain. The modulation filtering property is achieved by moving average (MA) modeling. The ARMA spectrograms are used to derive features for speech recognition in the Aurora-4 database. In these experiments, the ARMA model features provide significant improvements (relative improvements of 15%) compared to other robust features. Furthermore, the robustness of these features is also verified for language identification (LID) of highly degraded radio channel speech. Here, the ARMA approach achieves relative improvements of up to 20% over the baseline features.

Objective To establish an autoregression moving average (ARMA) model for predicting general traffic accidents and analyzing distributional difference on time series and frequency of common traffic accident so as to provide certain basis for a prediction model with better stability and accuracy. Methods The data of road traffic accidents in one newly developed zone of Chongqing in 2000-2005 were collected. The monthly distribution regularity of road traffic accidents was analyzed with descriptive epidemiologic method. ARMA model was set up for retrospective and prospective prediction. The predicted data were compared. Results Based on the characteristics of monthly distribution, the frequency of general traffic accidents in this area showed a cyclic fashion. The frequency of general traffic accidents predicted by ARMA model had over 80% of coincidence with the actual value. Conclusion The ARMA model can be used to predict the frequency of general traffic accidents, with better accuracy of short-term prediction than the long-term prediction. Key words: Accidents, traffic; ARMA; Time series; Prediction model

There is a lot of entrepreneurial competition in the production of goods or services in the world, especially in Indonesia, especially the production of staple goods, namely sugar. The problem that is often faced at Sugar Factory PTPN XI Semboro Jember is the lack of management that is neatly organized and efficient, which makes this company less working optimally. Often there is a lack and excess of sugar production which makes the sugar does not have the maximum value, the sugar has been damaged, and sales at a reduced price because the sugar is not as efficient as the initial product. From these various problems, it can reduce profits from the company. From these problems it can be concluded that the company needs a system that can organize the management of the company, and is able to forecast production in the future. In this research will make a forecasting system using the method of Autoregressive Integrated Moving Average (ARIMA), where this method is divided into three methods, namely the Autoregressive (AR) method, the Moving Average (MA) method, and the Autoregressive Integrated Moving Average (ARIMA) method, which preceded by checking stationary data, and modeling the Autoregressive Integrated Moving Average (ARIMA) method. Forecasting is done using production data for the previous 12 years from the company. The system is made to facilitate management that is less organized and displays predictions for the next production period. The results of this forecasting system are to determine the amount of production each year needed in this company. From the results of the ARIMA method modeling, the right ARIMA method is obtained by the ARIMA / AR (1,0,0), ARIMA / MA (0,0,1), and ARIMA (1,0,1) methods. The test results found that the average value of Mean Absolute Percentage Error (MAPE) in the Autoregressive (AR) method was 17%, the Moving Average (MA) method was 19%, and the Autoregressive Integrated Moving Average (ARIMA) method was 15%.

The performance of Autoregressive Moving Average and Multiple Linear Regression Models in predicting minimum and maximum temperatures of Ogun State is herein reported. Maximum and Minimum temperatures data covering a period of 29 years (1982 -2009) obtained from the Nigerian Meteorological Agency (NiMet), Abeokuta office, Nigeria, were used for the analyses. The data were first processed and aggregated into annual time series. Mann-Kendal non-parametric test and spectral analysis were carried out to detect whether there is trend, seasonal pattern, and either short or long memory in the time series. Mann-Kendal Z-values obtained are –0.47 and –2.03 for minimum and maximum temperatures respectively, indicating no trend, though the plot shows a slight change. The Lo’s R/S Q(N,q) values for minimum and maximum temperatures are 3.67 and 4.43, which are not within the range 0.809 and 1.862, thus signifying presence of long memory. The data was divided into two and the first 20 years data was used for model development, while the remaining was used for validation. Autoregressive Moving Average (ARMA) model of order (5, 3) and Autoregressive (AR) model of order 2 are found best for predicting minimum and maximum temperatures respectively. Multiple Linear Regression (MLR) model with 4 features (moving average, exponential moving average, rate of change and oscillator) were fitted for both temperatures. The ARMA and AR models were found to perform better with Mean Absolute Percentage Error (MAPE) values of -2.89 and -1.37 for minimum and maximum temperatures, compared with the Multiple Linear Regression Models with MAPE values of 141 and 876 respectively. Results of ARMA model can be relied on in generating forecast of temperature of the study area because of their minimal error values. However, it is recommended other climatic elements that were not captured in this paper due to unavailability of information be considered too in order to see which model is best for them.

OF THE DISSERTATION ON ORDER IDENTIFICATION OF TIME SERIES MODELS AND ITS APPLICATIONS by Shuhao Chen Dissertation Director: Rong Chen My thesis focuses on the order identification schemes of the widely-used time series model Autoregressive Integrated Moving-Average (ARIMA) model and the applications of the order determination methods. The first part investigates the impact of dependent but uncorrelated innovations (errors) on the traditional autoregressive integrated moving average (ARIMA) model order determination schemes such as autocorrelation function (ACF), partial autocorrelation function (PACF), extended autocorrelation function (EACF) and unit-root test. We also propose a new order determination scheme to address those impacts and can be used to time series sequences with uncorrelated innovations. In the second part, a unified approach for the tentative specification of both the seasonal and nonseasonal orders of general multiplicative seasonal model is proposed. This new approach has the advantages of determining the seasonal and nonseasonal orders simultaneously and automatically.

Whether or not a time series is weakly stationary has long been a question of major interest in the field of time series analysis. Stationary time series can be sufficiently described by means of autoregressive moving average (ARMA) processes. When modelling temporal correlations of GNSS observation noise, the applicability of ARMA processes depends on the stationarity of residual time series from GNSS data analysis. According to the property that stationary processes have homogenous variances, statistical inferences on stationarity can be made by testing for homogeneity of variance (HOV). In addition, considering a time series as a realisation of a stochastic process, stationarity can be assessed by testing for stochastic trends using unit root tests. Based on representative data simulations, this paper analyses the empirical size and power of commonly used HOV and unit root tests. The results show that the performance of the HOV test is strongly affected by serial correlations, whereas the unit root test produces high power without significant size distortions.

This study investigates the existing diurnal as well as night time surface ozone concentration trend over Delhi between 1990 and 2012. Secondary data obtained from the National Data Centre (NDC) of the India Meteorological Department (IMD) was analysed to assess the trend in the surface and night-time ozone concentration. This was further used to forecast the variation in the night-time ozone concentration over the city till 2025. A significantly increasing trend of the night-time ozone concentration was observed between 1990 and 2012 evidenced by a +0.158 value of the Mann Kendall test. Moreover, the forecasting of the variations conducted using the Autoregressive Integrated Moving Average (ARIMA) model revealed that the concentration of night-time ozone is expected to increase between the period of 2013 and 2025 if the current trend continues. This is the first study to conduct a trend analysis of night-time ozone concentration for a duration of three decades in the NCT of Delhi. Considering the negative impacts of elevated levels of ozone on the health status of individuals, agricultural productivity and air quality of the city, the present study highlights that it is imperative to take concentrated actions to curb the release of anthropogenic precursors of surface ozone.