LS estimation of periodic autoregressive models with non-Gaussian errors: a simulation study

The seemingly complex nature of river flow and the significant variability it exhibits in both time and space, have largely led to the development and application of the stochastic process concept for its modelling, forecasting, and other ancillary purposes. Towards this end, in this study, attempt was made at stochastic modelling of the daily streamflow process of the Benue River. In this regard, Autoregressive Moving Average (ARMA) models and its derivative, the Periodic Autoregressive (PAR) model were developed and used for forecasting. Comparative forecast performances of the different models indicate that despite the shortcomings associated with univariate time series, reliable forecasts can be obtained for lead times, 1 to 5 day-ahead. The forecast results also showed that the traditional ARMA model could not robustly simulate high flow regimes unlike the periodic AR (PAR). Thus, for proper understanding of the dynamics of the river flow and its management, especially, flood defense, in the light of this study, the traditional ARMA models may not be suitable since they do not allow for real-time appraisal. To account for seasonal variations, PAR models should be used in forecasting the streamflow processes of the Benue River. However, since almost all mechanisms involved in the river flow processes present some degree of nonlinearity thus, how appropriate the stochastic process might be for every flow series may be called to question.

Periodic autoregressive forecasting of global solar irradiation without knowledge-based model implementation

In this paper, we address the problem of modeling data with periodic autoregressive (PAR) time series and additive noise. In most cases, the data are processed assuming a noise-free model (i.e., without additive noise), which is not a realistic assumption in real life. The first two steps in PAR model identification are order selection and period estimation, so the main focus is on these issues. Finally, the model should be validated, so a procedure for analyzing the residuals, which are considered here as multidimensional vectors, is proposed. Both order and period selection, as well as model validation, are addressed by using the characteristic function (CF) of the residual series. The CF is used to obtain the probability density function, which is utilized in the information criterion and for residuals distribution testing. To complete the PAR model analysis, the procedure for estimating the coefficients is necessary. However, this issue is only mentioned here as it is a separate task (under consideration in parallel). The presented methodology can be considered as the general framework for analyzing data with periodically non-stationary characteristics disturbed by finite-variance external noise. The original contribution is in the selection of the optimal model order and period identification, as well as the analysis of residuals. All these findings have been inspired by our previous work on machine condition monitoring that used PAR modeling.

Inverse filtering based method for estimation of noisy autoregressive signals

This paper is concerned with identification of autoregressive (AR) model parameters using observations corrupted with colored noise. A novel formulation of an auxiliary least-squares estimator is introduced so that the autocovariance functions of the colored observation noise can be estimated in a straightforward manner. With this, the colored-noise-induced estimation bias can be removed to yield the unbiased estimate of the AR parameters. The performance of the proposed unbiased estimation algorithm is illustrated by simulation results. The presented work greatly extends the author's previous methods that were developed for identification of AR signals observed in white noise.

In many applications such as multiuser radar communications and astrophysical imaging processing, the encountered noise is usually described by the finite sum of -stable variables. In this paper, a new parameter estimator is developed, in the presence of this new heavy-tailed noise. Since the closed-form PDF of the -stable variable does not exist except and , we take the sum of the Cauchy () and Gaussian () noise as an example, namely, additive Cauchy-Gaussian (ACG) noise. The probability density function (PDF) of the mixed random variable, can be calculated by the convolution of the Cauchy's PDF and Gaussian's PDF. Because of the complicated integral in the PDF expression of the ACG noise, traditional estimators, e.g., maximum likelihood, are analytically not tractable. To obtain the optimal estimates, a new robust frequency estimator is devised by employing the Metropolis-Hastings (M-H) algorithm. Meanwhile, to guarantee the fast convergence of the M-H chain, a new proposal covariance criterion is also devised, where the batch of previous samples are utilized to iteratively update the proposal covariance in each sampling process. Computer simulations are carried out to indicate the superiority of the developed scheme, when compared with several conventional estimators and the Cramér-Rao lower bound.

Abduction is a kind of logical inference, and has been studied in computer science and artificial intelligence (Fin- lay and Dix 1996). Recently, Sawa and Gunji (2010) introduced a diagram to represent three types of inference: i.e. deduc- tion, induction, and abduction, which are articulated by C.S.Peirce. Sawa-Gunji’s representation provides a new approach to a numerical aspect of abduction. In the present paper, we show that Sawa-Gunji's representation of abduction is consistent with Finlay-Dix's one, and integrate the two representations. Both parameter estimation and abduction occupy a similar position on the integrated representation, although they are not completely corresponding. We present "incomplete" pa- rameter estimation as a sort of "simulated abduction", which is a numerical aspect of abduction. It is applied to a first-order autoregressive (AR(1)) model. As a result of numerical analyses on AR(1), the incompletely estimated parameter (IEP) follows a Cauchy distribution, which has a power law of the slope -2 in the tail, although conventionally estimated parameter is normally distributed. It is shown that the Cauchy distribution of the IEP is based on structure of ratio distribution of normal random variables generated from the AR(1). This research suggests that the distribution of the IEP is not based on a mech- anism of system itself, but on relationship between data structure on the given system (i.e. the given AR(1) process) and one on the system observer (i.e. the estimator of the AR(1) parameter).

The best accuracy for estimated spectra is obtained with parsimonious time series models, which have the smallest number of parameters to guarantee unbiased models. Durbin 's method for the estimation of moving average (MA) parameters uses the estimated parameters of a long autoregressive (AR) model to calculate MA parameters. Probably all pejorative remarks on the quality of Durbin's method in the literature are based on suboptimal or wrong choices for the method of AR estimation or for the order of the intermediate AR model. That AR order should be considerably higher than the order of the best predicting AR model and it should grow with the sample size. Furthermore, the Burg estimates for the AR parameters give the best results because they have the smallest variance of all AR methods with a small bias. The triangular autocorrelation bias of the popular Yule-Walker method of AR estimation can cause large bias errors in finite samples, which makes it unsuited. Durbin's method applied to the proper number of AR parameters estimated with Burg's method outperforms all other known MA estimation methods, asymptotically as well as in finite samples. The accuracy is generally close to the Cramer-Rao bound.

This paper presents a novel approach for estimating autoregressive (AR) model parameters using deep neural network (DNN) in the AR-Wiener filtering speech enhancement. Unlike conventional DNN that predicts one kind of target, the DNN used in this paper is trained to predict the AR model parameters of speech and noise simultaneously at offline stage. We train this network by minimizing the Euclidean distance between the output of DNN and the AR model parameters of clean speech and noise. At online stage, the acoustic features are first extracted from noisy speech as the input of the DNN. Then, AR model parameters of speech and noise are estimated by the DNN simultaneously. Finally, the Wiener filter is constructed by the AR model parameters of speech and noise. However, the AR model parameters only models the spectral shape not the spectral details, there are still some residual noise between the harmonics. In order to solve this problem, we introduce the speech-presence probability (SPP), that is, in the test stage, the SPP is estimated and is used to update the Wiener filter. The experimental results show that our approach has higher performance compared with some existing approaches.

In a Fourier transform infrared (IR) spectrometer, the Michelson interference signal extrapolation method based on linear prediction is often used to improve spectral resolution. In this method, an autoregressive (AR) model is established for the Michelson interference signal in the spectrometer. Once the AR model parameters are determined, the AR process is predictable. The interference signal can be used to figure out the AR model's parameters. Based on this, the AR model can be used to extrapolate the interference signal to improve the spectral resolution. In this paper, the forward-backward linear prediction total least squares (FB-TLS) method is proposed to estimate the parameters of the AR model. The parameters that are estimated are used to improve the IR spectral resolution. By simulating different order and signal-to-noise ratio situations, the effects of the Burg, the least square, and the FB-TLS parameter estimation methods on spectral resolution enhancement are studied. The simulation results demonstrate that the FB-TLS parameter estimation method can effectively suppress noise and avoid spurious peaks. The experimental results demonstrate that the FB-TLS parameter estimation method is effective for spectral resolution enhancement technology based on linear prediction. When the FB-TLS method is used to enhance NH3 IR spectral resolution from 2 cm-1 to 1 cm-1, the spectral prediction error in the NH3 characteristic band is only 0.21% compared with the measured NH3 spectrum, whose spectral resolution is 1 cm-1.

Lower order periodic stochastic models such as periodic autoregressive (PAR) and periodic autoregressive moving average (PARMA) models with and without harmonics are fitted to the monthly streamflows into Krishnarajasagara (KRS) reservoir located at the upper reaches of the river Cauvery in South India and a comparison is done between these models in terms of preservation of periodic statistics as well as over-year drought characteristics. Further, the comparison has been extended to the Thomas-Fiering lognormal 3-parameter (TFLN3) model which is supposed to be quite simple in structure and at the same time efficient in preservation of low flows. It is found that TFLN3 is able to reproduce the first two moments very well but, not the skewness or periodic correlations. On the other hand, PAR(2) and PARMA(1,1) models are able to preserve the periodic correlations. The periodic models which are not subjected to Fourier fitting, report lower relative root mean square error in overall standard deviation and correlations, compared to the ones which are subjected to Fourier fitting. From the over-year drought analysis, it is observed that all the stochastic models fitted underestimate severe drought durations corresponding to higher truncation levels. The over-year drought characteristics are computed using the method suggested by Wang and Salas (1989), based on one hundred periodic synthetic flow sequences generated from the different models. A comparative study of the preservation of drought characteristics corresponding to various truncation levels and return periods is presented.

New estimation methods for autoregressive process in the presence of white observation noise

The concept of fuzzy instrumental variable and an approach for fuzzy identification of nonlinear discrete time systems were proposed. Convergence conditions for identification in a noisy environment in a fuzzy context were studied. Simulation results for off-line and on-line schemes evidence the good quality of this fuzzy instrumental variable approach for identification and function approximation with observation errors in input and output data.

Maximum likelihood autoregressive model parameter estimation with noise corrupted independent snapshots

A Bayesian approach was used to fit a conceptual transpiration model to half‐hourly transpiration rates for a sugar maple (Acer saccharum) stand collected over a 5‐month period and probabilistically estimate its parameter and prediction uncertainties. The model used the Penman‐Monteith equation with the Jarvis model for canopy conductance. This deterministic model was extended by adding a normally distributed error term. This extension enabled using Markov chain Monte Carlo simulations to sample the posterior parameter distributions. The residuals revealed approximate conformance to the assumption of normally distributed errors. However, minor systematic structures in the residuals at fine timescales suggested model changes that would potentially improve the modeling of transpiration. Results also indicated considerable uncertainties in the parameter and transpiration estimates. This simple methodology of uncertainty analysis would facilitate the deductive step during the development cycle of deterministic conceptual models by accounting for these uncertainties while drawing inferences from data.