Time Series Forecasting with ARMA Models: A Practical Guide

Based on the time series analysis method, this article develops a Bayesian method of detecting and repairing the cycle slips in the GNSS carrier-phase data. Firstly, this article analyses the characteristics of the cycle slips in the GNSS carrier-phase observations and establishes the relationships between the cycle slips and the additive outliers (AOs) in the stationary time series. When the ARMA (autoregressive moving-average) model is used to fit the stationary time series obtained by differencing the GNSS carrier-phase observations, the detection of cycle slips in the GNSS carrier-phase observations can be transformed to the detection of AOs in the ARMA model. Then, this article proposes a Bayesian method of detecting the AOs in the ARMA model, and the implementation of detecting the cycle slips in the GNSS carrier-phase observations is also developed. Finally, the new Bayesian method of detecting the cycle slips is used to the real GNSS carrier-phase data. From the comparison among the Bayesian method, the high-order differences method and ionospheric residual method, we can find that the Bayesian method has a better detection efficiency for several kinds of cycle slips in the GNSS carrier-phase observations than other methods.

An autoregressive moving average (ARMA) is a time series model that is applied in everyday life for pattern recognition and forecasting. The ARMA model contains a noise which is assumed to have a specific distribution. The noise is often considered to have a Gaussian distribution. However in applications, the noise is sometimes found that does not have a Gaussian distribution. The first objective is to develop the ARMA model in which noise has a Laplacian distribution. The second objective is to estimate the parameters of the ARMA model. The ARMA model parameters include ARMA model orders, ARMA model coefficients, and noise variance. The parameter estimation of the ARMA model is carried out in the Bayesian framework. In the Bayesian framework, the ARMA model parameters are treated as a variable that has a prior distribution. The prior distribution for the ARMA model parameters is combined with the likelihood function for the data to get the posterior distribution for the parameter. The posterior distribution for parameters has a complex form so that the Bayes estimator cannot be determined analytically. The reversible jump Markov chain Monte Carlo (MCMC) algorithm was adopted to determine the Bayes estimator. The first result, the ARMA model can be developed by assuming Laplacian distribution noise. The second result, the performance of the algorithm was tested using simulation studies. The simulation shows that the reversible jump MCMC algorithm can estimate the parameters of the ARMA model correctly.

The quarterly electricity consumption curve shows certain regularity due to factors such as temperature and holidays. The ARMA(Autoregressive Moving Average) model has a good fitting ability for such time series, but the quarterly electricity consumption is also affected by economic growth. It is an ascending time series, which is a nonstationary time series. It cannot be directly predicted by ARMA model. Therefore, it is necessary to make finite difference processing to the original time series to stationary time series. The original time series is suitable for ARIMA(Autoregressive Integrated Moving Average) model. Any prediction model can not avoid the generation of residuals. The K-means clustering algorithm is used to divide the residual sequence state. By exploring the relationship between states at different moments, the transition matrix is established to predict the probability distribution of the residual state at the next moment. The predicted state is restored to the residual prediction value, and the prediction of the residual is completed to implement the correction of the original prediction value. That is, by predicting the residual by establishing a Markov model, the predicted value of the ARIMA model is corrected, and the prediction accuracy is improved.

Three important parametric models for describing the correlation functions and spectra of stationary stochastic processes are the autoregressive (AR), moving average (MA), and autoregressive-moving average (ARMA) models. Quite recently, the MATLAB toolbox “ARMASA” has been made publicly available. This toolbox provides state-of-the-art algorithms to perform automatic identification and selection between the models based on the estimated prediction error. ARMASA works on a single segment of data, whereas in some applications, the data are available as multiple segments. We could process each segment independently and average the estimated autocorrelation functions or spectra afterward. Better performance, however, can be expected when all segments are processed simultaneously, for two reasons. Initially, the bias in the estimated model parameters depends on the number of observations in a segment. Averaging models from segments does not reduce this bias, in contrast to identification from all the data at once. In either case, the variance of the estimated model parameters will decrease. By considering all the data at once this variance reduction can be considered in the order selection stage, decreasing the selection bias. In this paper, we extend the parameter estimation algorithms of the ARMASA toolbox to the case of multiple segments. AR parameter estimation from segments has been considered before, but not for the MA and ARMA models. In addition, in the AR case, model order selection can be done easily when the segments are independent. But, in the application we are primarily interested in, the segments are formed by lines of scanning transmission electron microscopy (STEM) images, which can be correlated with each other. Because the correlation among data segments influences the effective number of independent observations available it becomes necessary to estimate that number. This is not an easy problem in general. For observations from a stochastic process with an isotropic 2-D autocorrelation function we can, however, formulate an accurate estimator in terms of the line autocorrelation function. This situation arises, for example, in STEM images of amorphous specimens. From the AR model identification for segments, the extension to the MA and ARMA models is rather straightforward because MA and ARMA algorithms are based on Durbin's methods, whose first step is the estimation of a long AR model. We compare the segment-based algorithms to conventional ARMASA with simulations of stationary stochastic processes. An illustrative example involving simulated STEM images shows large potential benefits.

The seemingly complex nature of river flow and the significant variability it exhibits in both time and space, have largely led to the development and application of the stochastic process concept for its modelling, forecasting, and other ancillary purposes. Towards this end, in this study, attempt was made at stochastic modelling of the daily streamflow process of the Benue River. In this regard, Autoregressive Moving Average (ARMA) models and its derivative, the Periodic Autoregressive (PAR) model were developed and used for forecasting. Comparative forecast performances of the different models indicate that despite the shortcomings associated with univariate time series, reliable forecasts can be obtained for lead times, 1 to 5 day-ahead. The forecast results also showed that the traditional ARMA model could not robustly simulate high flow regimes unlike the periodic AR (PAR). Thus, for proper understanding of the dynamics of the river flow and its management, especially, flood defense, in the light of this study, the traditional ARMA models may not be suitable since they do not allow for real-time appraisal. To account for seasonal variations, PAR models should be used in forecasting the streamflow processes of the Benue River. However, since almost all mechanisms involved in the river flow processes present some degree of nonlinearity thus, how appropriate the stochastic process might be for every flow series may be called to question.

Time series events are counted within time intervals usually for convenience and practical reasons. Time series data involving counts frequently follows Poison distribution. Integrated Autoregressive Moving Average (ARIMA) models were studied on simulated Poison data of different parameter and sample sizes. These models were used to capture Poison data with different phenomena. Data set were simulated from Poison process with ARIMA (p, d, q) were then fitted to the simulated data so as to examine the effect of the changes in parameter value of the Poison on the models’ performance across the sample size. Thereafter, the same models were used to fit and forecast the daily Covid-19 data, confirmed cases from inception to February 2023. It was concluded that ARIMA (2, 1, 2) and ARIMA (1, 1, 1) are obviously the best Models at different sample sizes respectively. Based on the Covid-19 analysis, the estimated coefficient values of all ARIMA (p,d,q) strictly conforms to the bounds of the parameter between -1 and 1 and this had made the model to be stationary comparing the ARIMA models in terms of the AIC, BIC and MSE. It is clearly preferred ARIMA (1, 0, 2) to other models since their estimated AIC, BIC and MSE are smaller as compared to other models. The forecast is quite accurate, and hereby confirms what was expected. It seems that ARIMA (1, 0, 2) does a very good job in capturing the dynamic nature of the data and forecasting. It was observed that the confirmed cases increase, and then keep stationary throughout the future times.

In this paper, we proposed a new method able to select, simultaneously, from finite length time-series sequence, the suitable model type; pure Autoregressive (AR) or Autoregressive Moving Average (ARMA), which best describes the process that generated data. This is also a new way for estimating ARMA model order. The proposed method is different in its concept from all other methods found in the literature, direct and far simpler to implement. We proposed new empirical criteria basing on specific AR order selection criteria for finite length time series. We tested the proposed criteria on 10000 realizations of different broadband and narrow band processes. We also studied effects of additive noise and data length on their performance. The best mean recognition rate of model type was 92%. [Formula: see text]-value of Fisher’s exact test was always strictly under [Formula: see text]. Power spectral densities of selected models managed to recover the true Power spectral density (PSDs) shapes by resolving close spectral peaks and by finding their positions and their bandwidths with negligible spectral biases. Efficiency of our method was proved by its application in automatic segmentation of electrohysterographic signals (EHG) in the aim of predicting preterm deliveries with a mean rate of 90%.

Long memory time series and short term forecasts

The merits of the modelling philosophy of Box \& Jenkins (1970) are illustrated with a summary of our recent work on seasonal river flow forecasting. Specifically, this work demonstrates that the principle of parsimony, which has been questioned by several authors recently, is helpful in selecting the best model for forecasting seasonal river flow. Our work also demonstrates the importance of model adequacy. An adequate model for seasonal river flow must incorporate seasonal periodic correlation. The usual autoregressive-moving average (ARMA) and seasonal ARMA models are not adequate in this respect for seasonal river flow time series. A new diagnostic check, for detecting periodic correlation in fitted ARMA models is developed in this paper. This diagnostic check is recommended for routine use when fitting seasonal ARMA models. It is shown that this diagnostic check indicates that many seasonal economic time series also exhibit periodic correlation. Since the standard forecasting methods are inadequate on this account, it can be concluded that in many cases, the forecasts produced are sub-optimal. Finally, a limitation of the arbitrary combination of forecasts is also illustrated. Combining forecasts from an adequate parsimonious model with an inadequate model did not improve the forecasts whereas combining the two forecasts of two inadequate models did yield an improvement in forecasting performance. These findings also support the model building philosophy of Box \& Jenkins. The non-intuitive findings of Newbold \& Granger (1974) and Winkler \& Makridakis (1983) that the apparent arbitrary combination of forecasts from similar models will lead to forecasting performance is not supported by our case study with river flow forecasting.

Objective To establish an autoregression moving average (ARMA) model for predicting general traffic accidents and analyzing distributional difference on time series and frequency of common traffic accident so as to provide certain basis for a prediction model with better stability and accuracy. Methods The data of road traffic accidents in one newly developed zone of Chongqing in 2000-2005 were collected. The monthly distribution regularity of road traffic accidents was analyzed with descriptive epidemiologic method. ARMA model was set up for retrospective and prospective prediction. The predicted data were compared. Results Based on the characteristics of monthly distribution, the frequency of general traffic accidents in this area showed a cyclic fashion. The frequency of general traffic accidents predicted by ARMA model had over 80% of coincidence with the actual value. Conclusion The ARMA model can be used to predict the frequency of general traffic accidents, with better accuracy of short-term prediction than the long-term prediction. Key words: Accidents, traffic; ARMA; Time series; Prediction model

The application of time series analysis to describe the dynamic movements of suspension bridges

This paper uses the prediction method of time series analysis to fit and predict the stock daily logarithmic return series of the four major banks. The ARMA model and the GARCH model are constructed and compared empirically. The results show that the GARCH model is better than that of the ARMA model in the fitting effect. In the prediction effect, The ARMA model is the best, followed by the GARCH model.

This paper outlines how the stationary ARMA (p, q) model can be specified as a structural equations model. Maximum likelihood estimates for the parameters in the ARMA model can be obtained by software for fitting structural equation models. For pure AR and mixed ARMA models, these estimates are approximately unbiased, while the efficiency is as good as those of specialized recursive estimators. The reported standard errors are generally found to be valid. Depending on sample size, estimates for pure MA models are biased 5–10% and considerably less efficient. Some assets of the method are that ARMA model parameters can be estimated when only autocovariances are known, that model constraints can be incorporated, and that the fit between observed and modelled covariances can be tested by statistical methods. The method is applied to problems that involve the evaluation of pregnancy as a function of perceived bodily changes, the effect of policy interventions in crime prevention, and the influence of weather conditions on absence from work.

Univariate time-series analyses were conducted on stable carbon isotope ratios obtained from tree-ring cellulose. We looked for the presence and structure of autocorrelation. Significant autocorrelation violates the statistical independence assumption and biases hypothesis tests. Its presence would indicate the existence of lagged physiological effects that persist for longer than the current year. We analyzed data from 28 trees (60-85 years old; mean = 73 years) of western white pine (Pinus monticola Dougl.), ponderosa pine (Pinus ponderosa Laws.), and Douglas-fir (Pseudotsuga menziesii (Mirb.) Franco var. glauca) growing in northern Idaho. Material was obtained by the stem analysis method from rings laid down in the upper portion of the crown throughout each tree's life. The sampling protocol minimized variation caused by changing light regimes within each tree. Autoregressive moving average (ARMA) models were used to describe the autocorrelation structure over time. Three time series were analyzed for each tree: the stable carbon isotope ratio (delta(13)C); discrimination (delta); and the difference between ambient and internal CO(2) concentrations (c(a) - c(i)). The effect of converting from ring cellulose to whole-leaf tissue did not affect the analysis because it was almost completely removed by the detrending that precedes time-series analysis. A simple linear or quadratic model adequately described the time trend. The residuals from the trend had a constant mean and variance, thus ensuring stationarity, a requirement for autocorrelation analysis. The trend over time for c(a) - c(i) was particularly strong (R(2) = 0.29-0.84). Autoregressive moving average analyses of the residuals from these trends indicated that two-thirds of the individual tree series contained significant autocorrelation, whereas the remaining third were random (white noise) over time. We were unable to distinguish between individuals with and without significant autocorrelation beforehand. Significant ARMA models were all of low order, with either first- or second-order (i.e., lagged 1 or 2 years, respectively) models performing well. A simple autoregressive (AR(1)), model was the most common. The most useful generalization was that the same ARMA model holds for each of the three series (delta(13)C, delta, c(a) - c(i)) for an individual tree, if the time trend has been properly removed for each series. The mean series for the two pine species were described by first-order ARMA models (1-year lags), whereas the Douglas-fir mean series were described by second-order models (2-year lags) with negligible first-order effects. Apparently, the process of constructing a mean time series for a species preserves an underlying signal related to delta(13)C while canceling some of the random individual tree variation. Furthermore, the best model for the overall mean series (e.g., for a species) cannot be inferred from a consensus of the individual tree model forms, nor can its parameters be estimated reliably from the mean of the individual tree parameters. Because two-thirds of the individual tree time series contained significant autocorrelation, the normal assumption of a random structure over time is unwarranted, even after accounting for the time trend. The residuals of an appropriate ARMA model satisfy the independence assumption, and can be used to make hypothesis tests.

Autoregressive moving average (ARMA) models are useful statistical tools to examine the dynamical characteristics of ecological time-series data. Here, we illustrate the utility and challenges of applying ARMA (p,q) models, where p is the dimension of the autoregressive component of the model, and q is the dimension of the moving average component. We focus on parameter estimation and model selection, comparing both maximum likelihood (ML) and restricted maximum likelihood (REML) parameter estimation. While REML estimation performs better (has less bias) than ML estimation for ARMA (p,q) models with p = 1 (as has been found previously), for models with p > 1 the performance of the estimators is complicated by multimodal likelihood functions. The resulting difficulties in estimation lead to our recommendation that likelihood functions be routinely investigated when applying ARMA (p,q) models. To aid this investigation, we provide MATLAB and R code for the ML and REML likelihood functions. We further explore the consequences of measurement error, showing how it can be explicitly and implicitly incorporated into estimation. In addition to parameter estimation, we also examine model selection for identifying the correct model dimensions (p and q). Finally, we estimate the characteristic return rate of the stochastic process to its stationary distribution, a quantity that describes a key property of population dynamics, and investigate bias that results from both estimation and model selection. While fitting ARMA models to ecological time series with complex dynamics has challenges, these challenges can be surmounted, making ARMA a useful and broadly applicable approach.