Structural damage detection based on variational Bayesian inference and delayed rejection adaptive Metropolis algorithm

This paper presents a Bayesian estimation method for sequential direction finding. The proposed method estimates the number of directions of arrivals (DOAs) and their DOAs performing operations on the factor graph. The graph represents a statistical model for sequential beamforming. At each time step, belief propagation predicts the number of DOAs and their DOAs using posterior probability density functions (pdfs) from the previous time and a different Bernoulli-von Mises state transition model. Variational Bayesian inference then updates the number of DOAs and their DOAs. The method promotes sparse solutions through a Bernoulli-Gaussian amplitude model, is gridless, and provides marginal posterior pdfs from which DOA estimates and their uncertainties can be extracted. Compared to nonsequential approaches, the method can reduce DOA estimation errors in scenarios involving multiple time steps and time-varying DOAs. Simulation results demonstrate performance improvements compared to state-of-the-art methods. The proposed method is evaluated using ocean acoustic experimental data.

This study performs parameter inference in a partial differential equations system of pulmonary circulation. We use a fluid dynamics network model that takes selected parameter values and mimics the behaviour of the pulmonary haemodynamics under normal physiological and pathological conditions. This is of medical interest as it enables tracking the progression of pulmonary hypertension. We show how we make the fluids model tractable by reducing the parameter dimension from a 55D to a 5D problem. The Delayed Rejection Adaptive Metropolis algorithm, coupled with constraint non‐linear optimization, is successfully used to learn the parameter values and quantify the uncertainty in the parameter estimates. To accommodate for different magnitudes of the parameter values, we introduce an improved parameter scaling technique in the Delayed Rejection Adaptive Metropolis algorithm. Formal convergence diagnostics are employed to check for convergence of the Markov chains. Additionally, we perform model selection using different information criteria, including Watanabe Akaike Information Criteria.

Compressive sensing is a sub-Nyquist sampling technique for efficient signal acquisition and reconstruction of sparse or compressible signals. In order to account for the sparsity of the underlying signal of interest, it is common to use sparsifying priors such as Bernoulli–Gaussian-inverse Gamma (BGiG) and Gaussian-inverse Gamma (GiG) priors on the components of the signal. With the introduction of variational Bayesian inference, the sparse Bayesian learning (SBL) methods for solving the inverse problem of compressive sensing have received significant interest as the SBL methods become more efficient in terms of execution time. In this paper, we consider the sparse signal recovery problem using compressive sensing and the variational Bayesian (VB) inference framework. More specifically, we consider two widely used Bayesian models of BGiG and GiG for modeling the underlying sparse signal for this problem. Although these two models have been widely used for sparse recovery problems under various signal structures, the question of which model can outperform the other for sparse signal recovery under no specific structure has yet to be fully addressed under the VB inference setting. Here, we study these two models specifically under VB inference in detail, provide some motivating examples regarding the issues in signal reconstruction that may occur under each model, perform comparisons and provide suggestions on how to improve the performance of each model.

In-line inspection (ILI) data is commonly used in corrosion growth models (CGMs) to predict the corrosion growth in energy pipelines. A hierarchical stochastic corrosion growth model is considered in this paper which considers the variations in the corrosion growth, both spatially and temporally, the inherent measurement error of the ILI tools as well as the model uncertainties. These uncertainties are represented as unknown model variables and are often inferred using a Bayesian method [1], [2] and samples of the unknown parameters’ posterior probability density functions (PDFs) are obtained using Markov Chain Monte Carlo (MCMC) sampling techniques [3]. ILIs can result in massive data sets. In order for MCMC-based inference techniques to yield reasonably accurate results, many samples (approaching infinity) are required. This fact in addition to the massive data sets exponentially increases the scale of the inference problem from an attainable solution to a potentially impossible one that is limited by today’s computing power. For this reason, MCMC-based inference techniques can become inefficient in the cases where ILI datasets are large. The objective is to propose variational inference (VI) as an alternative to MCMC to determine a Bayesian solution for the unknown parameters in complex stochastic CGMs. VI produces approximations of the posterior PDFs by treating the inference as an optimization problem. Variational inference emerged from machine learning for Bayesian inference of large data sets; therefore, it is an appropriate tool to use in the analysis of mass pipeline inspection data[4]–[7]. This paper introduces VI to solve the inference problem and provide a solution for a hierarchical stochastic CGM to describe the defect-specific corrosion growth experienced in pipelines based on excessively large ILI datasets. To gauge the accuracy of the VI implementation in the model, the results are compared to a set of values generated using a stochastic gamma process that represents the corrosion growth process experienced by the pipe.

To improve the accuracy of multiple target tracking in the clutter environment, a new joint probabilistic data association (JPDA) algorithm based on variational Bayesian adaptive moment estimation is proposed. First, considering the existence of measurements, the posterior distribution of the target state in JPDA is composed of two parts of probability weighting, that is, the posterior distribution of the target state that the real measurement exists in the association gate and the posterior distribution of the target state that the real measurement does not exist in the association gate. By combining the conjugate properties of the prior and posterior distributions, the prior distributions of the target state in the two cases are classified to provide more accurate a priori information to filter, so as to improve the accuracy of data association. Second, considering the coupling effect between state estimation and data association process, combined with variational Bayesian inference, the problem of minimizing Kullback–Leibler divergence is transformed into the problem of maximizing the evidence lower bound, thereby effectively measuring the distance between the posterior distribution of target state estimation and the real posterior distribution, so as to improve the accuracy of data association again from the perspective of optimizing nonlinear filter. Finally, the adaptive momentum estimation strategy is introduced to iteratively solve the variable distribution that meets the maximization of the evidence lower bound, and the optimization of the posterior distribution of the target state is completed. Theoretical derivation and simulation experiments are conducted to verify the feasibility and effectiveness of the algorithm.

In this letter, we propose a novel ship detection method in synthetic aperture radar (SAR) imagery via variational Bayesian inference. First, we establish the ship detection probabilistic model which decomposes the SAR image as the sum of a sparse component associated with ships and a sea clutter component. Then, we introduce hierarchical priors of the latent variables in the model and use variational Bayesian inference to estimate the posterior distributions of the latent variables. The proposed method is an automatic iterative process without any sliding window. Experimental results accomplished over synthetic data and a RADARSAT-2 SAR image demonstrate that the proposed method can achieve state-of-the-art ship detection performance.

Uncertainty propagation using infinite mixture of Gaussian processes and variational Bayesian inference

We combine inverse autoregressive flows (IAF) and variational Bayesian inference (variational Bayes) in the context of geophysical inversion parameterized with deep generative models encoding complex priors. Variational Bayes approximates the unnormalized posterior distribution parametrically within a given family of distributions by solving an optimization problem. Although prone to bias if the chosen family of distributions is too limited, it provides a computationally-efficient approach that scales well to high-dimensional inverse problems. To enhance the expressiveness of the variational distribution, we explore its combination with IAFs that allow samples from a simple base distribution to be pushed forward through a series of invertible transformations onto an approximate posterior. The IAF is learned by maximizing the lower bound of the evidence (marginal likelihood), which is equivalent to minimizing the Kullback–Leibler divergence between the approximation and the target posterior distribution. In our examples, we use either a deep generative adversarial network (GAN) or a variational autoencoder (VAE) to parameterize complex geostatistical priors. Although previous attempts to perform Gauss–Newton inversion in combination with GANs of the same architecture were proven unsuccessful, the trained IAF provides a good reconstruction of channelized subsurface models for both GAN- and VAE-based inversions using synthetic crosshole ground-penetrating-radar data. For the considered examples, the computational cost of our approach is seven times lower than for Markov chain Monte Carlo (MCMC) inversion. Furthermore, the VAE-based approximations in the latent space are in good agreement. The VAE-based inversion requires only one sample to estimate gradients with respect to the IAF parameters at each iteration, while the GAN-based inversions need more samples and the corresponding posterior approximation is less accurate.

Variational approximations provide fast, deterministic alternatives to Markov chain Monte Carlo for Bayesian inference on the parameters of complex, hierarchical models. Variational approximations are often limited in practicality in the absence of conjugate posterior distributions. Recent work has focused on the application of variational methods to models with only partial conjugacy, such as in semiparametric regression with heteroscedastic errors. Here, both the mean and log variance functions are modeled as smooth functions of covariates. For this problem, we derive a mean field variational approximation with an embedded Laplace approximation to account for the nonconjugate structure. Empirical results with simulated and real data show that our approximate method has significant computational advantages over traditional Markov chain Monte Carlo; in this case, a delayed rejection adaptive Metropolis algorithm. The variational approximation is much faster and eliminates the need for tuning parameter selection, achieves good fits for both the mean and log variance functions, and reasonably reflects the posterior uncertainty. We apply the methods to log-intensity data from a small angle X-ray scattering experiment, in which properly accounting for the smooth heteroscedasticity leads to significant improvements in posterior inference for key physical characteristics of an organic molecule.

SUMMARYWe test a fully non-linear method to solve Bayesian seismic tomographic problems using data consisting of observed traveltimes of first-arriving waves. Rather than using Monte Carlo methods to sample the posterior probability distribution that embodies the solution of the tomographic inverse problem, we use variational inference. Variational methods solve the Bayesian inference problem under an optimization framework by seeking the best approximation to the posterior distribution from a family of distributions, while still providing fully probabilistic results. We introduce a new variational method for geophysics—normalizing flows. The method models the posterior distribution by using a series of invertible and differentiable transforms—the flows. By optimizing the parameters of these transforms the flows are designed to convert a simple and analytically known probability distribution into a good approximation of the posterior distribution. Numerical examples show that normalizing flows can provide an accurate tomographic result including full uncertainty information while significantly decreasing the computational cost compared to Monte Carlo and other variational methods. In addition, this method provides analytic solutions for the posterior distribution rather than an ensemble of posterior samples. This opens the possibility that subsequent calculations that use the posterior distribution might be performed analytically.

Geoscientists use observed data to estimate properties of the Earth's interior. This often requires non‐linear inverse problems to be solved and uncertainties to be estimated. Bayesian inference solves inverse problems under a probabilistic framework, in which uncertainty is represented by a so‐called posterior probability distribution. Recently, variational inference has emerged as an efficient method to estimate Bayesian solutions. By seeking the closest approximation to the posterior distribution within any chosen family of distributions, variational inference yields a fully probabilistic solution. It is important to define expressive variational families so that the posterior distribution can be represented accurately. We introduce boosting variational inference (BVI) as a computationally efficient means to construct a flexible approximating family comprising all possible finite mixtures of simpler component distributions. We use Gaussian mixture components due to their fully parametric nature and the ease with which they can be optimized. We apply BVI to seismic travel time tomography and full waveform inversion, comparing its performance with other methods of solution. The results demonstrate that BVI achieves reasonable efficiency and accuracy while enabling the construction of a fully analytic expression for the posterior distribution. Samples that represent major components of uncertainty in the solution can be obtained analytically from each mixture component. We demonstrate that these samples can be used to solve an interrogation problem: to assess the size of a subsurface target structure. To the best of our knowledge, this is the first method in geophysics that provides both analytic and reasonably accurate probabilistic solutions to fully non‐linear, high‐dimensional Bayesian full waveform inversion problems.

Many human genes express multiple transcript isoforms through alternative splicing, which greatly increases diversity of protein function. Although RNA sequencing (RNA-Seq) technologies have been widely used in measuring amounts of transcribed mRNA, accurate estimation of transcript isoform abundances from RNA-Seq data is challenging because reads often map to more than one transcript isoforms or paralogs whose sequences are similar to each other. We propose a statistical method to estimate transcript isoform abundances from RNA-Seq data. Our method can handle gapped alignments of reads against reference sequences so that it allows insertion or deletion errors within reads. The proposed method optimizes the number of transcript isoforms by variational Bayesian inference through an iterative procedure, and its convergence is guaranteed under a stopping criterion. On simulated datasets, our method outperformed the comparable quantification methods in inferring transcript isoform abundances, and at the same time its rate of convergence was faster than that of the expectation maximization algorithm. We also applied our method to RNA-Seq data of human cell line samples, and showed that our prediction result was more consistent among technical replicates than those of other methods. An implementation of our method is available at http://github.com/nariai/tigar nariai@megabank.tohoku.ac.jp Supplementary data are available at Bioinformatics online.

This letter addresses the recovery of hierarchical sparse signals in a Bayesian framework. Hierarchical sparse signals exhibit two levels of sparsity, i.e., block-sparsity among different blocks and internal sparsity within each individual block. As in sparse Bayesian learning, each component of the coefficient vector is firstly modeled as a Gaussian distributed variable with zero mean. To enforce the two-level hierarchical sparsity, the variance is further modeled by two classes of hidden variables controlling the block-sparsity and the internal sparsity, respectively. Finally, variational Bayesian inference is used to recover the coefficient vector from the noise corrupted data. Numerical simulation and experimental results show that the proposed method outperforms those recently reported recovery methods.

We propose two generic methods for improving semi-supervised learning (SSL). The first integrates weight perturbation (WP) into existing “consistency regularization” (CR) based methods. We implement WP by leveraging variational Bayesian inference (VBI). The second method proposes a novel consistency loss called “maximum uncertainty regularization” (MUR). While most consistency losses act on perturbations in the vicinity of each data point, MUR actively searches for “virtual” points situated beyond this region that cause the most uncertain class predictions. This allows MUR to impose smoothness on a wider area in the input-output manifold. Our experiments show clear improvements in classification errors of various CR based methods when they are combined with VBI or MUR or both.

A low-complexity algorithm based on variational Bayesian inference for MIMO channel estimation