Sequential Monte Carlo for Cut-Bayesian Posterior Computation

Data assimilation is the process of estimating the state of dynamic systems (linear or nonlinear, Gaussian or non-Gaussian) as accurately as possible from noisy observational data. Although the Three Dimensional Variational (3D-VAR) methods, Four Dimensional Variational (4D-VAR) methods and Ensemble Kalman filter (EnKF) methods are widely used and effective for linear and Gaussian dynamics, new methods of data assimilation are required for the general situation, that is, nonlinear non-Gaussian dynamics. General Bayesian recursive estimation theory is reviewed in this thesis. The Bayesian estimation approach provides a rather general and powerful framework for handling nonlinear, non-Gaussian, as well as linear, Gaussian estimation problems. Despite a general solution to the nonlinear estimation problem, there is no closed-form solution in the general case. Therefore, approximate techniques have to be employed. In this thesis, the sequential Monte Carlo (SMC) methods, commonly referred to as the particle filter, is presented to tackle non-linear, non-Gaussian estimation problems. In this thesis, we use the SMC methods only for the nonlinear state estimation problem, however, it can also be used for the nonlinear parameter estimation problem. In order to demonstrate the new methods in the general nonlinear non-Gaussian case, we compare Sequential Monte Carlo (SMC) methods with the Ensemble Kalman Filter (EnKF) by performing data assimilation in nonlinear and non-Gaussian dynamic systems. The models used in this study are referred to as state-space models. The Lorenz 1963 and 1966 models serve as test beds for examining the properties of these assimilation methods when used in highly nonlinear dynamics. The application of Sequential Monte Carlo methods to different fixed parameters in dynamic models is considered. Four different scenarios in the Lorenz 1063 [sic] model and three different scenarios in the Lorenz 1996 model are designed in this study for both the SMC methods and EnKF method with different filter siz

The regular and repetitive nature of the sequential Monte Carlo (SMC) method makes it very attractive for implementation using parallel and pipelined architectures. This paper develops a VLSI architecture for the hardware implementation of the SMC algorithm using bootstrap filter. A flat fading wireless channel is considered as our framework on which the channel estimator is designed and implemented using the SMC method. The design and verification activities at the algorithm level, architecture level, and the circuit level are reviewed. The proposed architecture is verified with an FPGA implementation.

Assimilating real-time sensor data into large-scale spatial-temporal simulations, such as simulations of wildfires, is a promising technique for improving simulation results. This asks for advanced data assimilation methods that can work with the complex structures and nonlinear behaviors associated with the simulation models. This article presents a data assimilation framework using Sequential Monte Carlo (SMC) methods for wildfire spread simulations. The models and algorithms of the framework are described, and experimental results are provided. This work demonstrates the feasibility of applying SMC methods to data assimilation of wildfire spread simulations. The developed framework can potentially be generalized to other application areas where sophisticated simulation models are used.

Online Estimation of DSGE Models

In several implementations of Sequential Monte Carlo (SMC) methods it is natural and important, in terms of algorithmic efficiency, to exploit the information of the history of the samples to optimally tune their subsequent propagations. In this article we provide a carefully formulated asymptotic theory for a class of such adaptive SMC methods. The theoretical framework developed here will cover, under assumptions, several commonly used SMC algorithms [Chopin, Biometrika 89 (2002) 539-551; Jasra et al., Scand. J. Stat. 38 (2011) 1-22; Schafer and Chopin, Stat. Comput. 23 (2013) 163- 184]. There are only limited results about the theoretical underpinning of such adaptive methods: We will bridge this gap by providing a weak law of large numbers (WLLN) and a central limit theorem (CLT) for some of these algorithms. The latter seems to be the first result of its kind in the literature and provides a formal justification of algorithms used in many real data contexts [Jasra et al. (2011); Schafer and Chopin (2013)]. We establish that for a general class of adaptive SMC algorithms [Chopin (2002)], the asymptotic variance of the estimators from the adaptive SMC method is identical to a limiting SMC algorithm which uses ideal proposal kernels. Our results are supported by application on a complex high-dimensional posterior distribution associated with the Navier-Stokes model, where adapting highdimensional parameters of the proposal kernels is critical for the efficiency of the algorithm.

We consider estimating a stochastic intensity process, indirectly observed through a sequence of Poisson random variables. The class of dynamical systems we consider, are autoregressive state processes and a sequence of Poisson distributed observations, each influenced by the state process. Computing explicit recursive filters for this class of system can be technically difficult. Further, exact filters for this class of model do not have fixed memory requirements. It is shown in Manton et al. (1999), that the exact filter for these models has linear growth with respect to time in the number of sufficient statistics needed to construct the filter density. Some approximate suboptimal filters are available for these models using Edgeworth expansions. Using Sequential Monte Carlo (SMC) methods and the so called Bayesian bootstrap filter, we propose a scheme to estimate an intensity process for the models just described. The scheme we present is simple to implement and has fixed memory requirements. A simulation study is included to demonstrate the performance of SMC methods for discrete time models with Poisson observations.

Intractable sequential inference arises in several applications in the physical engineering process. One of the most successful and efficient stochastic simulation techniques that is appropriate to perform the Bayesian inference (inversion decision) in complex dynamic models is the sequential Monte Carlo (SMC) methods. Although, these methods have become widely accepted among researchers and practitioners, in practice the manifestation of the path degeneracy implies a poor approximation of the target measures. This is an unavoidable deep problem when approximating sequence target models, and it becomes evident when strong non-linearity is presented. To deal with this drawback of the SMC methods, a promise mechanism is used to create particle diversity to move the particle according some kernel perturbation. In this article, we apply in three non-linear engineering dynamical systems (reservoir dynamics, van de Vusse reaction and pipe-in-pipe), a class of move algorithms, without the need to change the computational complexity called the move-regweighting scheme proposed by Marques and Storvik [(Particle move-reweighting strategies for online inference. Preprint series. Statistical Research Report (1), 2013]. Numerical simulations with different number of particles are performed to demonstrate the accuracy of the algorithm.

In statistical signal processing, the sequential Monte Carlo (SMC) method is powerful and can approach the theoretical optima. However, its computational complexity is usually very high, especially in multiple-input multiple-output (MIMO) systems. This paper presents a new low-complexity SMC (LC-SMC) algorithm for blind detection in MIMO systems, the main idea of which is to shrink the sampling space via channel estimation which is initialized using the first differentially modulated symbol and then updated using the Monte Carlo samples. Since the a posteriori probability of the transmitted symbols can be calculated separately by each transmit antenna, the proposed LC-SMC algorithm is not only computationally efficient, as compared to the original SMC whose complexity grows exponentially with the number of transmit antennas, but also makes blind turbo receiver more feasible for multilevel/phase modulations. Simulation results are presented to demonstrate the effectiveness of the LC-SMC algorithm

Sequential Monte Carlo methods for complexity-constrained MAP equalization of dispersive MIMO channels

The requirement for Terrain Referenced Navigation (TRN) also known as Terrain Aided Navigation (TAN) arises when Global Navigation Satellite System (GNSS) signals are unavailable, jammed or blocked. In recent years, research on the application of TRN to aerial and underwater vehicles has been increased rapidly with the developments in the accuracy of digital terrain elevation database (DTED). Since the land and sea floor profiles are inherently nonlinear, TRN becomes a nonlinear estimation problem. Because of the highly nonlinear and non-Gaussian problem, linear or linearized estimation techniques such as Kalman or Extended Kalman Filter (EKF) do not work properly for many terrain profiles. Hence, this paper focuses on the Sequential Monte Carlo (SMC) method namely the particle filter (PF) for dealing with nonlinearities and different types of probability distributions even multi modal. Two different PF implementations are studied, Sequential Importance Sampling with effective resampling (SIS-R) and Sampling Importance Resampling (SIR). The major difference between these implementations is that in SIS-R algorithm, resampling is only performed using effective sample size when actually required. Therefore, the SIS-R algorithm has less computational demands than the SIR algorithm for a given number of particles. Both algorithms are tested for an aircraft scenario over a commercial map. Simulations with different number of particles and inertial measurement units (IMUs) having various error measures are performed and investigated. Monte Carlo (MC) simulations show that both algorithms yield similar results in terms of horizontal position accuracy and convergence. Hereby with this paper, it is presented that besides requiring less computational load, the SIS-R implementation works accurately on TRN problem.

We investigate two novel blind receivers for joint channel estimation and data detection in flat-fading multiple-input multiple-output (MIMO) communication systems. Both schemes are based on the sequential Monte Carlo (SMC) methodology, also known as particle filtering, and attain asymptotically optimal performance, both in terms of channel estimation and data detection. The most salient feature of the new SMC receivers is that their computational complexity growth is polynomial in the number of input data streams, in contrast to the complexity of most previously proposed SMC algorithms for blind MIMO equalization, which increases exponentially with the number of input streams. We present simulation results to show that the reduction in complexity is attained with a negligible impact on performance.

In this article we introduce a new proposal distribution to be used in conjunction with the sequential Monte Carlo (SMC) method of solving non-linear filtering problem. The proposal distribution incorporates all the information about the to be estimated current state form both the available state and observation processes. This makes it more effective than the state transition density which is more commonly used but ignores the recent observation. Because of its Gaussian nature it is also very easy to implement. We show further that the introduced proposal performs better than other similar importance functions which also incorporate both state and observations.

Particle filtering, also known as sequential Monte Carlo (SMC) sampling, has been successfully applied to general state−space models for Bayesian inference. Being a simulation method, its performance relies to some extent on the generated samples or particles. For a poor initial guess a large fraction of particles is usually less representative of the underlying state's distribution and could even cause SMC to diverge. In this paper, an intuitive statistic, predictive density is proposed to monitor the particles' performance. When below a statistically controlled threshold value, our approach triggers smoothing for obtaining a better estimate of the initial state in the case of a poor prior. We find that combining a moving horizon smoother with SMC is very effective for recovering from a poor prior and develop an integrated practical approach that combines these two powerful tools.

In recent years sequential Monte Carlo (SMC) methods have been applied to handle some of the problems inherent to model-based tracking. Two issues regarding SMC are investigated in the context of estimating the 3D pose of the human arm. Firstly, we investigate how to apply a subspace to representing the pose of a human arm more efficiently, i.e., reducing the dimensionality. Secondly, we investigate how to apply a local method to estimated the maximum a posteriori (MAP). The former issue is based on combining a screw axis representation with the position of the hand in the image. The latter issue is handled by applying a method based on maximising a proximity function, to estimate the MAP. We find that both the subspace and the proximity function are sound strategies and that they are an improvement over the current SMC-methods.

International audience