Maximum Likelihood Estimation for Periodic Autoregressive Moving Average Models

ABSTRACTResults involving correlation properties and parameter estimation for autoregressive‐moving average models with periodic parameters are presented. A multivariate representation of the PARMA model is used to derive parameter space restrictions and difference equations for the periodic autocorrelations. Close approximation to the likelihood function for Gaussian PARMA processes results in efficient maximum‐likelihood estimation procedures. Terms in the Fourier expansion of the parameters are sequentially included, and a selection criterion is given for determining the optimal number of harmonics to be included. Application of the techniques is demonstrated through analysis of a monthly streamflow time series.

This paper presents a constructive neural network model for seasonal streamflow forecasting. This surface water hydrology is basic to the design and operation of the reservoir. A good example is the operation of a reservoir with an uncontrolled inflow but having a means of regulating the outflow. If information on the nature of the inflow is determinable in advance, then the reservoir can be operated by some decision rule to minimize downstream flood damage. For this reasons, several companies in the Brazilian Electrical Sector use the linear time-series models such as PARMA (Periodic Auto regressive Moving Average) models developed by Box-Jenkins. This paper provides for river flow prediction a numerical comparison between neural networks, called nonlinear sigmoidal regression blocks networks (NSRBN) and PARMA models. The model was implemented to forecast weekly average inflow on an step-ahead basis. It was tested on four hydroelectric plants located in different river basins in Brazil. The results obtained in the evaluation of the performance of NSRBN were better than the results obtained with PARMA models.

An autoregressive moving average (ARMA) is a time series model that is applied in everyday life for pattern recognition and forecasting. The ARMA model contains a noise which is assumed to have a specific distribution. The noise is often considered to have a Gaussian distribution. However in applications, the noise is sometimes found that does not have a Gaussian distribution. The first objective is to develop the ARMA model in which noise has a Laplacian distribution. The second objective is to estimate the parameters of the ARMA model. The ARMA model parameters include ARMA model orders, ARMA model coefficients, and noise variance. The parameter estimation of the ARMA model is carried out in the Bayesian framework. In the Bayesian framework, the ARMA model parameters are treated as a variable that has a prior distribution. The prior distribution for the ARMA model parameters is combined with the likelihood function for the data to get the posterior distribution for the parameter. The posterior distribution for parameters has a complex form so that the Bayes estimator cannot be determined analytically. The reversible jump Markov chain Monte Carlo (MCMC) algorithm was adopted to determine the Bayes estimator. The first result, the ARMA model can be developed by assuming Laplacian distribution noise. The second result, the performance of the algorithm was tested using simulation studies. The simulation shows that the reversible jump MCMC algorithm can estimate the parameters of the ARMA model correctly.

Three important parametric models for describing the correlation functions and spectra of stationary stochastic processes are the autoregressive (AR), moving average (MA), and autoregressive-moving average (ARMA) models. Quite recently, the MATLAB toolbox “ARMASA” has been made publicly available. This toolbox provides state-of-the-art algorithms to perform automatic identification and selection between the models based on the estimated prediction error. ARMASA works on a single segment of data, whereas in some applications, the data are available as multiple segments. We could process each segment independently and average the estimated autocorrelation functions or spectra afterward. Better performance, however, can be expected when all segments are processed simultaneously, for two reasons. Initially, the bias in the estimated model parameters depends on the number of observations in a segment. Averaging models from segments does not reduce this bias, in contrast to identification from all the data at once. In either case, the variance of the estimated model parameters will decrease. By considering all the data at once this variance reduction can be considered in the order selection stage, decreasing the selection bias. In this paper, we extend the parameter estimation algorithms of the ARMASA toolbox to the case of multiple segments. AR parameter estimation from segments has been considered before, but not for the MA and ARMA models. In addition, in the AR case, model order selection can be done easily when the segments are independent. But, in the application we are primarily interested in, the segments are formed by lines of scanning transmission electron microscopy (STEM) images, which can be correlated with each other. Because the correlation among data segments influences the effective number of independent observations available it becomes necessary to estimate that number. This is not an easy problem in general. For observations from a stochastic process with an isotropic 2-D autocorrelation function we can, however, formulate an accurate estimator in terms of the line autocorrelation function. This situation arises, for example, in STEM images of amorphous specimens. From the AR model identification for segments, the extension to the MA and ARMA models is rather straightforward because MA and ARMA algorithms are based on Durbin's methods, whose first step is the estimation of a long AR model. We compare the segment-based algorithms to conventional ARMASA with simulations of stationary stochastic processes. An illustrative example involving simulated STEM images shows large potential benefits.

In this paper we study the weighted least absolute deviations estimator (WLADE) for Periodic ARMA (PARMA) models with unspecified noises, asymptotic normality of the estimator is derived. The random weighting approach is proposed to estimate the asymptotic covariance matrices of WLADE. Simulation study is carried out to examine the performance of the proposed procedure.

Seasonality is a distinctive characteristic which is often observed in many practical time series. Artificial Neural Networks (ANNs) are a class of promising models for efficiently recognizing and forecasting seasonal patterns. In this paper, the Particle Swarm Optimization (PSO) approach is used to enhance the forecasting strengths of feedforward ANN (FANN) as well as Elman ANN (EANN) models for seasonal data. Three widely popular versions of the basic PSO algorithm, viz. Trelea-I, Trelea-II and Clerc-Type1 are considered here. The empirical analysis is conducted on three real-world seasonal time series. Results clearly show that each version of the PSO algorithm achieves notably better forecasting accuracies than the standard Backpropagation (BP) training method for both FANN and EANN models. The neural network forecasting results are also compared with those from the three traditional statistical models, viz. Seasonal Autoregressive Integrated Moving Average (SARIMA), Holt-Winters (HW) and Support Vector Machine (SVM). The comparison demonstrates that both PSO and BP based neural networks outperform SARIMA, HW and SVM models for all three time series datasets. The forecasting performances of ANNs are further improved through combining the outputs from the three PSO based models.

This paper proposes an identification method of ARIMA models for seasonal time series using an intermediary model and a filtering method. This method is found to be useful when conventional methods, such as using sample ACF and PACF, fail to reveal a clear-cut model. This filtering identification method is also found to be particularly effective when a seasonal time series is subjected to calendar variations, moving-holiday effects, and interventions.

Lower order periodic stochastic models such as periodic autoregressive (PAR) and periodic autoregressive moving average (PARMA) models with and without harmonics are fitted to the monthly streamflows into Krishnarajasagara (KRS) reservoir located at the upper reaches of the river Cauvery in South India and a comparison is done between these models in terms of preservation of periodic statistics as well as over-year drought characteristics. Further, the comparison has been extended to the Thomas-Fiering lognormal 3-parameter (TFLN3) model which is supposed to be quite simple in structure and at the same time efficient in preservation of low flows. It is found that TFLN3 is able to reproduce the first two moments very well but, not the skewness or periodic correlations. On the other hand, PAR(2) and PARMA(1,1) models are able to preserve the periodic correlations. The periodic models which are not subjected to Fourier fitting, report lower relative root mean square error in overall standard deviation and correlations, compared to the ones which are subjected to Fourier fitting. From the over-year drought analysis, it is observed that all the stochastic models fitted underestimate severe drought durations corresponding to higher truncation levels. The over-year drought characteristics are computed using the method suggested by Wang and Salas (1989), based on one hundred periodic synthetic flow sequences generated from the different models. A comparative study of the preservation of drought characteristics corresponding to various truncation levels and return periods is presented.

Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir

The author considers serial correlation testing in seasonal time series models. He proposes a test statistic based on a spectral approach. Many tests of this type rely on kernel‐based spectral density estimators that assign larger weights to low order lags than to high ones. Under seasonality, however, large autocorrelations may occur at seasonal lags that classical kernel estimators cannot take into account. The author thus proposes a test statistic that relies on the spectral density estimator of Shin (2004), whose weighting scheme is more adapted to this context. The distribution of his test statistic is derived under the null hypothesis and he studies its behaviour under fixed and local alternatives. He establishes the consistency of the test under a general fixed alternative. He also makes recommendations for the choice of the smoothing parameters. His simulation results suggest that his test is more powerful against seasonality than alternative procedures based on classical weighting schemes. He illustrates his procedure with monthly statistics on employment among young Americans.

This paper studies correlation and partial autocorrelation properties of periodic autoregressive moving‐average (PARMA) time series models. An efficient algorithm to compute PARMA autocovariances is first derived. An innovations based algorithm to compute partial autocorrelations for a general periodic series is then developed. Finally, periodic moving averages and autoregressions are characterized as periodically stationary series whose autocovariances and partial autocorrelations, respectively, are zero at all lags that exceed some periodically varying threshold.

Autoregressive moving average models of crown profiles for two California hardwood species

For analysis and design of water resource systems, it is sometimes useful to generate high-resolution (e.g., weekly) synthetic river flows. Periodic autoregressive moving average (PARMA) time series models provide a powerful tool for generating synthetic flows. Periodically stationary models are indicated when the basic statistics (mean, variance, and autocorrelation) of the time series exhibit significant seasonal variations. Parameter estimation for high-resolution PARMA models involves numerous parameters, which can lead to overfitting. Thus, this paper develops a parsimonious method of parameter fitting for high-resolution PARMA models, using discrete Fourier transforms to represent the set of periodic autoregressive and moving average model coefficients. Model parameters are computed via the innovations algorithm, and the asymptotic distributions of the discrete Fourier transform coefficients are obtained. Those asymptotic results are useful to determine the statistically significant Fourier coeffici...

We propose a portmanteau test for autoregressive moving average (ARMA) time series models with infinite variance. Our test is based on the partial autocorrelation function (PACF) of trimmed residuals from the fitted model. The asymptotic null distribution of the test statistic is derived when the innovation has a Pareto-type distribution and the estimators of the ARMA coefficients satisfy certain condition. The effectiveness of our test is established through extensive simulation studies. In general, our test agrees very well with that of Lee and Ng (2010) and under some circumstances it performs even better. Three real world examples are also provided to demonstrate its usage.

Abstract. The aim of this article is to give a complete description of the spectral measure of periodic autoregressive moving‐average (PARMA) system in terms of its coefficients. In the analysis we use the spectral theory of strongly harmonizable sequences presented in Hurd [Journal of Multivariate Analysis(1989) Vol. 29, pp. 53–67] and the form of the unique bounded solution of ARMA model with periodic coefficients. As an application of the theoretical results, we present some examples of the spectral measures for PARMA models.