Introduction to state space time series analysis

Time variant reliability model updating in instrumented dynamical systems based on Girsanov’s transformation

Deep learning has been used to train neural networks to estimate the Remaining Useful Life (RUL) of a machine given sensor signals from that machine. This has resulted in some accurate state-of-the-art RUL estimation models. Here we present a novel way to construct a generative model to estimate the RUL and quantify its uncertainty. It takes the form of a Linear Gaussian State Space Model (LGSSM) and is trained using the Kalman Filter; hence, uncertainty is quantified using this LGSSM instead of by other popular methods like Monte Carlo Dropout or Deep Ensembles. This means we can train by directly using the marginal log-likelihood loss and don’t require multiple samples to represent the uncertainty. However, this method is limited to using Gaussian distributions to quantify uncertainty. We avoid needing to use non-linear variants of the filter by processing the sensors using a neural network to represent noncausal sensor sequences as a “control variable” in the LGSSM. This noncausal representation is shown to be important for achieving state-of-the-art performance. The model is tested on a turbofan engine and dust filter dataset. The code can be found on GitHub. 1 1 https://github.com/StarMarco/Kalman_DVAE • Deep Neural Networks encode time window of sensor signals. • These are used as control/forcing variables of a Linear Gaussian State Space Model (LGSSM). • The “observables" of the LGSSM are the Remaining Useful Life (RUL) values of the machine. • The Kalman filtering algorithm is used to estimate the marginal log-likelihood of these RUL estimates. • After training the LGSSM can be evaluated over time to estimate the RUL trajectories.

Application of a Linear Gaussian State Space Model and the Kalman Filter in Estimating Expectations of the Natural Logarithmic Monthly Nav Returns of the Open – End Funds of Major Investment Banks.

In time-series analysis, state-space models (SSMs) have been widely used to estimate the conditional probability distributions of hidden variables and parameter values, as well as to understand structures that can generate the data. For example, the Kalman filter is used to analytically calculate the conditional probability distribution on linear SSMs in terms of minimizing the variance, and several extensions, such as the unscented Kalman filter and particle filter, have been applied to calculate the approximate distribution on nonlinear SSMs. Recently, the approximate Bayesian computation (ABC) has been applied to such time-series filtering to handle intractable likelihoods; however, it remains problematic with respect to consistently achieving a reduction of the estimation bias, evaluating the validity of the models, and dealing with replicated observations. To address these problems, in this paper, we propose a novel method combined with the kernel ABC to perform filtering, parameter estimation, and the model evaluation in SSMs. Simulation studies show that the proposed method produces inference that is comparable to other ABC methods, with the advantage of not requiring a careful calibration of the ABC threshold. In addition, we evaluate the performance of the model-selection capability using true and competitive models on synthetic data from nonlinear SSMs. Finally, we apply the proposed method to real data in rat circadian oscillations, and demonstrated the usefulness in practical situations.

This tutorial provides a gentle introduction to the particle Metropolis-Hastings (PMH) algorithm for parameter inference in nonlinear state-space models together with a software implementation in the statistical programming language R. We employ a step-by-step approach to develop an implementation of the PMH algorithm (and the particle filter within) together with the reader. This final implementation is also available as the package pmhtutorial from the Comprehensive R Archive Network (CRAN) repository. Throughout the tutorial, we provide some intuition as to how the algorithm operates and discuss some solutions to problems that might occur in practice. To illustrate the use of PMH, we consider parameter inference in a linear Gaussian state-space model with synthetic data and a nonlinear stochastic volatility model with real-world data.

A new algorithm for the prediction, filtering, and smoothing of non-Gaussian nonlinear state space models is shown. The algorithm is based on a Monte Carlo method in which successive prediction, filtering (and subsequently smoothing), conditional probability density functions are approximated by many of their realizations. The particular contribution of this algorithm is that it can be applied to a broad class of nonlinear non-Gaussian higher dimensional state space models on the provision that the dimensions of the system noise and the observation noise are relatively low. Several numerical examples are shown.

Nudging is a popular algorithmic strategy in numerical filtering to deal with the problem of inference in high-dimensional dynamical systems. We demonstrate in this paper that general nudging techniques can also tackle another crucial statistical problem in filtering, namely the misspecification of the transition kernel. Specifically, we rely on the formulation of nudging as a general operation increasing the likelihood and prove analytically that, when applied carefully, nudging techniques implicitly define state-space models that have higher marginal likelihoods for a given (fixed) sequence of observations. This provides a theoretical justification of nudging techniques as data-informed algorithmic modifications of state-space models to obtain robust models under misspecified dynamics. To demonstrate the use of nudging, we provide numerical experiments on linear Gaussian state-space models and a stochastic Lorenz 63 model with misspecified dynamics and show that nudging offers a robust filtering strategy for these cases.

The problem of sequential detection of spontaneous changes in equations for unobserved state variables and observations of linear and nonlinear multivariate state-space models is considered. We consider the case of new additive terms and the case of changing coefficients of these equations. The proposed nonparametric method uses the idea of the “moving window” of observations. In Theorem 4.1 the exponential rate of convergence to zero for type 1 error probability (a wrong decision about a change) is proved. In Theorem 4.2 we consider type 2 error probability and the normalized (by the volume N of the moving window) delay time in change-point detection. Asymptotic optimality of the proposed method is proved in Theorem 5.1. Experimental study includes Monte Carlo examples of sequentially detected changes in coefficients of linear and nonlinear state-space models.

Sequential Monte Carlo (SMC) methods (also known as particle filter) provide a way to solve the state estimation problem in nonlinear non-Gaussian state space models (SSM) through numerical approximation. Particle smoothing is one retrospective state estimation method based on particle filtering. In this paper, we propose a new particle smoother. The basic idea is easy and leads to a forward-backward procedure, where the Metropolis-Hastings algorithm is used to resample the filtering particles. The goodness of the new scheme is assessed using a nonlinear SSM. It is concluded that this new particle smoother is suitable for state estimation in complicated dynamical systems.

This paper focuses on sequential Monte Carlo approximations of smoothing distributions in conditionally linear and Gaussian state spaces. To reduce Monte Carlo variance of smoothers, it is typical in these models to use Rao-Blackwellization: particle approximation is used to sample sequences of hidden regimes while the Gaussian states are explicitly integrated conditional on the sequence of regimes and observations, using variants of the Kalman filter/smoother. The first successful attempt to use Rao-Blackwellization for smoothing extends the Bryson-Frazier smoother for Gaussian linear state space models using the generalized two-filter formula together with Kalman filters/smoothers. More recently, a forward-backward decomposition of smoothing distributions mimicking the Rauch-Tung-Striebel smoother for the regimes combined with backward Kalman updates has been introduced. This paper investigates the benefit of introducing additional rejuvenation steps in all these algorithms to sample at each time instant new regimes conditional on the forward and backward particles. This defines particle-based approximations of the smoothing distributions whose support is not restricted to the set of particles sampled in the forward or backward filter. These procedures are applied to commodity markets which are described using a two-factor model based on the spot price and a convenience yield for crude oil data.

Lidar technology is used to quantify airborne particulate matter less than 10 μm in diameter (PM10). These spatio-temporal lidar data on PM10 are subject to censoring due to detection limits. A non-linear and non-Gaussian state-space model is modified to accommodate data subject to detection limits and outline strategies for Markov-chain Monte Carlo estimation and filtering. The methods are applied to spatio-temporal lidar measurements of dust particle concentrations. Copyright © 2004 John Wiley & Sons, Ltd.

Model estimation from spatio-temporal data is important topic since it helps us to extract useful information from big data in recent years. In this paper, we introduce an estimation algorithm of the linear Gaussian state space model with focusing on the real-time property. The proposed algorithm uses two key ideas, localization and spatial uniformity, to reduce the number of the parameters. Thanks to this, we obtain stable method to estimate the parameters regarding state transition and states. In addition, the proposed algorithm is quicker and more accurate than existing methods, therefore, it suffices the requirement of the rapid response for the alternation of the fields.

<p>By modelling and forecasting  of meteorological  time  series it is possible to  improve   understanding  of  the  weather dynamics and fluctuations as a result of climate change . The most frequently used forecasting models are exponential smoothing, ARIMA models (Box and Jenkins, 1970), state-space models (Harvey, 1989) and innovations State Space Models (Hyndman et al., 2008).</p><p>The aim of this study was to check the effectiveness of the coupled TBATS and Support Vector Machines (SVM) model, supplied with some measured meteorological quantities to forecast air temperature for six years for four climatic localizations in Europe. The study was calculated from northern (Jokioinen in Finland), central (Dikopshof located in the west part of Germany and Nossen in the south part of Germany) and southern (Lleida in Spain) Europe to present different climatic conditions. Jokioinen city has a subarctic climate that has severe winters, with cool and short summers and strong seasonality. Lleida has a semi-arid climate with Mediterranean. Dikopshof represents maritime temperate climate. There are significant precipitation throughout the year in Dikopshof and Nossen. In the study we study on air temperature dataset collected on a daily basis from January 1st 1980 to December 31st 2010 (11322 days).</p><p>For all the studied sites coupled TBATS/SVM models occurred to be effective in predicting air temperature courses, giving an improved precision (up to 25%) in forecasting of the seasonality and local temperature variations, compared to pure SVM or TBATS modelling. The precision of prediction of the maximum and minimum air temperatures strongly depended on the dynamics of the weather conditions, and varied for different climatic zones.</p><p>This study has been partly financed from the funds of the Polish National Centre for Research and Development in frame of the project: MSINiN, contract number: BIOSTRATEG3/343547/8/NCBR/2017.</p><p> </p><p>Reference to a journal publication:</p><p>BOX, G.E.P. – Jenkins, G. 1970. Time Series Analysis: forecasting and control. Holden-Day, p. 20-31.</p><p>HARVEY A. 1989. Forecasting Structural Time Series Model and the Kalman Filter. New York, Cambridge University press., p. 32-41.</p><p>HYNDMAN, R.J. – KOEHLER, A.B. – ORD, J.K. – SNYDER, R.D. 2008. Forecasting with Exponential Smoothing: The State Space Approach. Springer-Verlag, p. 50-62.</p>

In this study, identification of nonlinear aerodynamic systems based on an improved nonlinear state-space modeling approach is presented to predict transonic aeroelastic behaviors of aircraft structures. It starts with identifying a linear state-space aerodynamic model by using the eigensystem realization algorithm and observer/Kalman filter identification methods. Subsequently, the remaining parameters of the nonlinear state-space aerodynamic model are determined through an output error-minimization procedure, based on a new training data generation methodology. To illustrate the approach, two- and three-dimensional transonic aeroelastic configurations are studied. The transonic aeroelastic behaviors predicted via the proposed approach agree well with those obtained via computational fluid dynamics technique in the interested range of dynamic pressure.

Particle filter-based Gaussian process optimisation for parameter inference