Bayesian model assessment and comparison using cross-validation predictive densities.

In the context of Bayesian inversion for scientific and engineering modeling, Markov chain Monte Carlo (MCMC) sampling strategies have become the benchmark due to their flexibility and robustness in dealing with arbitrary posterior probability density functions (PDFs). However, these algorithms have been shown to be inefficient when sampling from high-dimensional posterior distributions or exhibit multimodality and/or strong parameter correlations. In such contexts, transitional MCMC (TMCMC) provides a more efficient alternative. Despite the recent applicability for Bayesian updating and model selection across a variety of disciplines, TMCMC may require a prohibitive number of tempering stages when the prior pdf is significantly different from the target posterior. Furthermore, the need to start with an initial set of samples from the prior distribution may present a challenge when dealing with implicit priors, e.g., based on feasible regions. Finally, TMCMC cannot be used for inverse problems with improper prior PDFs that represent a lack of prior knowledge on all or a subset of parameters. A generalization of TMCMC is proposed that alleviates such challenges and limitations toward providing a more robust and efficient tempering sampling strategy. We present convergence analysis, proving that the distance between the intermediate distributions and the target posterior distribution monotonically decreases as the algorithm proceeds. We also demonstrate the advantages of the proposed generalization through a series of test problems and an engineering application in the oil and gas industry.

The dynamic nature of the ocean environment presents numerous challenges to parameter estimation in ocean acoustics. We present a variational Bayesian method which selects an optimal model for parameter estimation in ocean acoustics using Gaussian process (GP) regression. GP model selection is based on Bayesian model selection, which treats the parameters and kernel selection of the model as hyperparameters and uses Bayesian inference to compute a model’s likelihood with respect to its hyperparameters. However, in Bayesian model selection, the integrals over parameter space are often intractable and require approximation or sampling methods such as Markov chain Monte Carlo. GP renders these integrals tractable by treating hyperparameters as multivariate Gaussian distributions, which greatly speeds up the optimization. Furthermore, model selection using GPs allows for straightforward evaluation of uncertainty in the selected model’s likelihood. In this study, we compare results of acoustic parameter estimation from Bayesian and GP model selection using an ocean acoustic propagation model.

Bayesian model selection for nonlinear aeroelastic systems using wind-tunnel data

Complexity Analysis of Multilayer Perceptron Neural Network Embedded into a Wireless Sensor Network

Global illumination algorithms using Markov chain Monte Carlo (MCMC) sampling are well-known for their efficiency in scenes with complex light transport. Samples in such algorithms are generated as a history of Markov chain states so that they are distributed according to the contributions to the image. The whole process is done based only on the information of the path contributions and user-defined transition probabilities from one state to the others. In light transport simulation, however, there is more information that can be used to improve the efficiency of path sampling. A notable example is multiple importance sampling (MIS) in bidirectional path tracing, which utilizes the probability densities of constructing a given path with different estimators. While MIS is a powerful ordinary Monte Carlo method, how to incorporate such additional information into MCMC sampling has been an open problem. We introduce a novel MCMC sampling framework, primary space serial tempering , which fuses the ideas of MCMC sampling and MIS for the first time. The key idea is to explore not only the sample space using a Markov chain, but also different estimators to generate samples by utilizing the information already available for MIS. Based on this framework, we also develop a novel rendering algorithm, multiplexed Metropolis light transport , which automatically and adaptively constructs paths with appropriate techniques as predicted by MIS. The final algorithm is very easy to implement, yet in many cases shows comparable (or even better) performance than significantly more complex MCMC rendering algorithms.

This chapter provides a brief introduction to the use of Bayesian graphical models in R. In these models, parameters are treated as random quantities on an equal footing with the random variables. This allows complex stochastic systems to modeled, often using Markov chain Monte Carlo (MCMC) sampling methods. We first consider a series of examples, ranging from simple repeated sampling, linear regression models, random coefficient regression models, to the chest clinic example of Chap. 3. We formulate these as Bayesian graphical models, and represent them graphically in a compact form using plates. We then describe a special case, in which the unknown parameters are all discrete, and explain how probability propagation methods described in Chap. 3 may be used to compute the posterior distributions. We then turn to the general case, when MCMC sampling is required, and explain the computations involved in Metropolis-Hastings sampling and Gibbs sampling. Finally we illustrate a Bayesian linear regression analysis using the R2WinBUGS package.

This paper presents a new methodology for reliability-based inspection planning focusing on robust and accurate computational strategies for fatigue-reliability updating using inspection results. The core of the proposed strategy is a conditioned sampling-based method, implemented by a Fast Probability Analyzer (FPA) software where efficiency is achieved by using the importance sampling principal. For a single component or limit state, FPA first generates Markov-Chain Monte Carlo (MCMC) samples in the failure domain, then applies an adaptive stratified importance sampling (ASIS) method to compute probability of failure (PoF) with error control. Once the MCMC samples have been created, solving a reliability updating problem is fairly straightforward and computationally robust relative to the conventional system reliability methods that rely on linearization of the limit states. The new approach is demonstrated using examples including stiffened panels of a ship-shaped vessel where reliability is updated using inspection results from 100 panel connections.

This study aimed to establish the correlation between activated partial thromboplastin time (aPTT) in seconds and ratio, first derivative peak (Max1), second derivative peak (Max2), second derivative minimum peak (Min2), and delta-OD of clot waveform analysis in patients diagnosed with cirrhosis, sepsis, and a control group. The study involved a series of cases using the Bayesian ANOVA model with Bayes factor (BF) estimated using the Markov Chain Monte Carlo sampling method with 2 chains. Each chain consisted of 10 000 repetitions and started 5 000 times to predict the variables aPTT (s), aPTT (ratio), Max1, Max2, Min2, and delta-OD respectively across the 3 study groups. The model used was variable ~ stage – 1. This analysis illustrates the adjustment of the posterior proportion for multiple testing by setting the prior probability of the initial hypothesis to 0.5 for all comparisons. There is insufficient evidence that aPTT (s), aPTT (ratio), and Min2 differ between the study groups (0 < BF < 3). There is compelling evidence for the Max1 value in both the sepsis group and the control group (BF10 = 44.042). Additionally, between the sepsis group and the cirrhosis group, there is substantial evidence for the Max1 value (BF10 = 5.842). Moreover, there is convincing evidence for the Max2 value in the sepsis group compared to the control group (BF10 = 38.782). There is almost confirmatory evidence that the delta-OD value in the sepsis group is higher than that in the control group (BF10 = 1251.781). Finally, there is convincing evidence that the sepsis group has a higher delta-OD value than the cirrhosis group (BF10 = 35.555). The Max1, Max2, and delta-OD indices effectively reflect coagulation disorders in patients with cirrhosis and sepsis, even when the aPTT test results are within normal limits.

Ciprian M. Crainiceanu 'Likelihood ratio testing for zero variance components in linear mixed effects models.' - Daowen Zhang & Xihong Lin 'Variance component testing in generalized linear mixed models for longitudinal/clustered data and other related topics.' -Satkartar Kinney & David Dunson 'Bayesian model uncertainty in mixed effects models.' -Bo Cai & David Dunson 'Bayesian variable selection in generalized linear mixed models.' - Peter M. Bentler & Jiajuan Liang 'A unified approach to two-level structural equation models and linear mixed effects models.' - Sik-Yum Lee & Xin-Yuan Song 'Bayesian model comparison of structural equation models.' - Joyee Ghosh & David Dunson 'Bayesian model selection in factor analytic models.'

Bayesian analysis of spatial dynamic panel data model with convex combinations of different spatial weight matrices: A reparameterized approach

Entity resolution is the process of merging and removing duplicate records from multiple data sources, often in the absence of unique identifiers. Bayesian models for entity resolution allow one to include a priori information, quantify uncertainty in important applications, and directly estimate a partition of the records. Markov chain Monte Carlo (MCMC) sampling is the primary computational method for approximate posterior inference in this setting, but due to the high dimensionality of the space of partitions, there are no agreed upon standards for diagnosing nonconvergence of MCMC sampling. In this article, we review Bayesian entity resolution, with a focus on the specific challenges that it poses for the convergence of a Markov chain. We review prior methods for convergence diagnostics, discussing their weaknesses. We provide recommendations for using MCMC sampling for Bayesian entity resolution, focusing on the use of modern diagnostics that are commonplace in applied Bayesian statistics. Using simulated data, we find that a commonly used Gibbs sampler performs poorly compared with two alternatives.

The creep deformation process is constructed by complex interactions of multiple factors, and the measurement of creep deformation requires enormous economic costs and a long experimental time, so there is a small amount of measurement data. In such a situation, multiple models are often proposed to explain the same experimental data. The coexistence of multiple models based on different physical assumptions makes it difficult to understand the creep deformation process. The purpose of this study is to construct a framework to compare and evaluate coexistence models based on measurement data using the Bayesian model selection framework. Basically, in the creep deformation model, basis functions, usually the exponential function eat or power function tb, corresponding to the primary creep stage and the tertiary creep stage are set, and the time series of the whole creep deformation process is regressed as the linear sum of these bases. The parameters a and b of the basis functions are parameters that must also be optimized to regress the time series data. Izuno et al. proposed the Bayesian model selection method for the creep deformation process, and it was confirmed that the model with the linear term corresponding to the secondary creep stage, called the modified theta method, is a good model compared to the model without adding the linear term. To obtain the criterion of Bayesian model selection analytically, which can reduce the computational cost greatly, Izuno et al. treated only the regression coefficients as the probabilistic parameter of the creep model. In this method, parameters a and b of the basis functions are optimized by a grid search. Because the grid search requires significant computational costs if multiple basis functions are set for each creep stage, as in the model proposed by Kimura et al., it is difficult to apply the algorithm to models with multiple bases. In addition, if parameters a and b of the basis functions correlate with the regression coefficients, the method of Izuno et al. would offer a biased evaluation of the models. To improve such an existing method, we proposed a Bayesian model selection method that treats the parameters of the basis functions as probabilistic parameters. The large computational cost to treat the parameters of the basis functions as probabilistic parameters was reduced using the exchange Monte Carlo method, which is an efficient sampling method. By applying our proposed method to the evaluation of the creep deformation model, information was obtained to understand the creep deformation process deeply. For example, it was confirmed that there exists a model in which there is a correlation between the parameter of basis function and the regression coefficients.

Small datasets, which are common in many traditional science and engineering disciplines, present a challenge for powerful machine learning methods that are better suited for large datasets. Through the example of spontaneous colloidal sol-gel transition, we present a data-driven framework based on Gaussian Processes (GPs) for modeling and inference with limited data. GPs are well-suited for small datasets and automatically provide uncertainty quantification for model predictions. We explore GPs with two different structures (single and multi-output) to map synthesis parameters to gelation characteristics. While both GP structures quantitatively capture the relationships between the input and output variables, we favor multioutput GP, which attempts to simultaneously learn correlations between different outputs, due to its superior uncertainty calibration. To address inverse problems, we introduce a Bayesian framework that embeds a GP model into a Markov Chain Monte Carlo sampling algorithm. It enables the discovery of multiple synthesis pathways for achieving desired material properties. Therefore, this work presents a robust methodology for data-constrained modeling, with broad applicability to material design in general and the colloidal domain in particular.

This paper presents a consistent Bayesian solution for data integration and history matching for oil reservoirs while accounting for both model and parameter uncertainties. The developed method uses Gaussian Process Regression to build a permeability map conforming to collected data at well bores. Following that, an augmented Markov Chain Monte Carlo sampler is used to condition the permeability map to dynamic production data. The selected proposal distribution for the Markov Chain Monte Carlo conforms to the Gaussian process regression output. The augmented Markov Chain Monte Carlo sampler allows transition steps between different models of the covariance function, and hence both the parameter and model space are effectively explored. In contrast to single model Markov Chain Monte Carlo samplers, the proposed augmented Markov Chain Monte Carlo sampler eliminates the selection bias of certain covariance structures of the inferred permeability field. The proposed algorithm can be used to account for general model and parameter uncertainties.

Subglacial drainage models sensitively depend on the values of numerous uncertain parameters. However, the computation time associated with running these models makes it difficult to quantify the associated uncertainty in model outputs and to use field data to calibrate parameter values. To overcome these computational limitations, we construct a Gaussian Process (GP) emulator that accelerates subglacial drainage modelling by ~1000x. The GP predicts spatiotemporally resolved water pressure as a function of eight model parameters and is trained using ensembles of up to 512 simulations with the Glacier Drainage System (GlaDS) model applied to the Kangerlussuaq sector of the western Greenland Ice Sheet. The GP reproduces the spatial patterns and daily temporal variations simulated by GlaDS within ~4%, with locally higher errors near moulins and during the early melt season. As an application of the GP, we compute the sensitivity of basal water pressure to each of the eight parameters and find that three parameters (ice-flow coefficient, bed bump aspect ratio and the subglacial cavity system conductivity) explain 90% of the variance in model outputs. Next, we explore using a borehole water-pressure timeseries to calibrate the eight uncertain parameters. We take a Bayesian perspective to quantify the uncertainty in parameter estimates and use the GP in place of the physics-based model to make Markov Chain Monte Carlo sampling computationally feasible. We find meaningful constraints relative to the prior assumptions on most parameters and a factor-of-three reduction in uncertainty of the calibrated model predictions. However, significant differences between the calibrated model and the borehole data suggest that structural limitations of the model, rather than poorly constrained parameters or computational cost, remain the most important constraint on subglacial drainage modelling.