Efficient Monte Carlo resampling for probability measure changes from Bayesian updating

Bayesian methods are rapidly becoming popular tools for making statistical inference in various fields of science including biology, engineering, finance, and genetics. One of the key aspects of Bayesian inferential method is its logical foundation that provides a coherent framework to utilize not only empirical but also scientific information available to a researcher. Prior knowledge arising from scientific background, expert judgment, or previously collected data is used to build a prior distribution which is then combined with current data via the likelihood function to characterize the current state of knowledge using the so-called posterior distribution. Bayesian methods allow the use of models of complex physical phenomena that were previously too difficult to estimate (e.g., using asymptotic approximations). Bayesian methods offer a means of more fully understanding issues that are central to many practical problems by allowing researchers to build integrated models based on hierarchical conditional distributions that can be estimated even with limited amounts of data. Furthermore, advances in numerical integration methods, particularly those based on Monte Carlo methods, have made it possible to compute the optimal Bayes estimators. However, there is a reasonably wide gap between the background of the empirically trained scientists and the full weight of Bayesian statistical inference. Hence, one of the goals of this chapter is to bridge the gap by offering elementary to advanced concepts that emphasize linkages between standard approaches and full probability modeling via Bayesian methods.

The swift progression and expansion of machine learning (ML) have not gone unnoticed within the realm of statistical mechanics. In particular, ML techniques have attracted attention by the classical density-functional theory (DFT) community, as they enable automatic discovery of free-energy functionals to determine the equilibrium-density profile of a many-particle system. Within classical DFT, the external potential accounts for the interaction of the many-particle system with an external field, thus, affecting the density distribution. In this context, we introduce a statistical-learning framework to infer the external potential exerted on a classical many-particle system. We combine a Bayesian inference approach with the classical DFT apparatus to reconstruct the external potential, yielding a probabilistic description of the external-potential functional form with inherent uncertainty quantification. Our framework is exemplified with a grand-canonical one-dimensional classical particle ensemble with excluded volume interactions in a confined geometry. The required training dataset is generated using a Monte Carlo (MC) simulation where the external potential is applied to the grand-canonical ensemble. The resulting particle coordinates from the MC simulation are fed into the learning framework to uncover the external potential. This eventually allows us to characterize the equilibrium density profile of the system by using the tools of DFT. Our approach benchmarks the inferred density against the exact one calculated through the DFT formulation with the true external potential. The proposed Bayesian procedure accurately infers the external potential and the density profile. We also highlight the external-potential uncertainty quantification conditioned on the amount of available simulated data. The seemingly simple case study introduced in this work might serve as a prototype for studying a wide variety of applications, including adsorption, wetting, and capillarity, to name a few.

A unified approach for time-invariant and time-variant reliability-based design optimization with multiple most probable points

Markov chainMonteCarlo (MCMC)methodshavebeenaround for almost as longasMonte Carlo techniques, even though their impact on statistics was not truly felt until the very early 1990s, except in the specialized fields of spatial statistics and image analysis, where those methods appeared earlier. The emergence of Markov based techniques in physics is a story that remains untold within this survey (see Landau and Binder, 2005). Also, we will not enter into a description of MCMC techniques, unless they have some historical link, as the remainder of this volume covers the technical aspects.Acomprehensive treatment with further references can also be found in Robert and Casella (2004). We will distinguish between the introduction of Metropolis-Hastings based algorithmsand those related to Gibbs sampling, since they each stem from radically different origins, even though their mathematical justification via Markov chain theory is the same. Tracing the development of Monte Carlo methods, we will also briefly mention what we might call the “second-generation MCMC revolution.” Starting in the mid to late 1990s, this includes the development of particle filters, reversible jump and perfect sampling, and concludes with more current work on population or sequential Monte Carlo and regeneration and the computing of “honest” standard errors. As mentioned above, the realization that Markov chains could be used in a wide varietyof situations only came (tomainstream statisticians)withGelfand and Smith (1990), despite earlier publications in the statistical literature such as Hastings (1970), Geman and Geman (1984), and Tanner and Wong (1987). Several reasons can be advanced: lack of computing machinery (think of the computers of 1970!), or background onMarkov chains, or hesitation to trust in the practicality of themethod. It thus required visionary researchers like Gelfand and Smith to convince the community, supported by papers that demonstrated, through a series of applications, that the method was easy to understand, easy to implement and practical (Gelfand et al., 1990, 1992; Smith and Gelfand, 1992; Wakefield et al., 1994). The rapid emergence of the dedicated BUGS (Bayesian inference using Gibbs sampling) software as early as 1991, when a paper on BUGS was presented at the Valencia meeting, was another compelling argument for adopting, at large, MCMC algorithms.∗Monte Carlo methods were born in Los Alamos, New Mexico, during World War II, eventually resulting in theMetropolis algorithm in the early 1950s. WhileMonte Carlo methods were in use by that time, MCMC was brought closer to statistical practicality by the work of Hastings in the 1970s. What can be reasonably seen as the first MCMC algorithm is what we now call theMetropolis algorithm, published by Metropolis et al. (1953). It emanates from the same group of scientists who produced the Monte Carlo method, namely the research scientists of Los Alamos, mostly physicists working on mathematical physics and the atomic bomb. MCMC algorithms therefore date back to the same time as the development of regular(MC only) Monte Carlo methods, which are usually traced to Ulam and von Neumann in the late 1940s. StanislawUlam associates the original ideawith an intractable combinatorial computation he attempted in 1946 (calculating the probability of winning at the solitaire card game). This ideawas enthusiastically adopted by John vonNeumann for implementationwith direct applications to neutron diffusion, the name “Monte Carlo” being suggested by Nicholas Metropolis. Eckhardt (1987) describes these early Monte Carlo developments, and Hitchcock (2003) gives a brief history of the Metropolis algorithm. These occurrences very closely coincide with the appearance of the very first general-purpose digital computer, the ENIAC,which came to life in February 1946, after three years of construction. The Monte Carlo method was set up by von Neumann, who was using it on thermonuclear and fission problems as early as 1947. That same year, Ulam and vonNeumann invented inversion andaccept-reject techniques (also recounted inEckhardt, 1987) to simulate from nonuniform distributions. Without computers, a rudimentary version invented by Fermi in the 1930s went unrecognized (Metropolis, 1987). Note also that, as early as 1949, a symposiumonMonteCarlowas supported byRand, theNational Bureau of Standards, and theOakRidge laboratory and thatMetropolis andUlam (1949) published the very first paper about the Monte Carlo method.

In a Bayesian setting, inverse problems and uncertainty quantification (UQ)—the propagation of uncertainty through a computational (forward) model—are strongly connected. In the form of conditional expectation the Bayesian update becomes computationally attractive. This is especially the case as together with a functional or spectral approach for the forward UQ there is no need for time-consuming and slowly convergent Monte Carlo sampling. The developed sampling-free non-linear Bayesian update is derived from the variational problem associated with conditional expectation. This formulation in general calls for further discretisation to make the computation possible, and we choose a polynomial approximation. After giving details on the actual computation in the framework of functional or spectral approximations, we demonstrate the workings of the algorithm on a number of examples of increasing complexity. At last, we compare the linear and quadratic Bayesian update on the small but taxing example of the chaotic Lorenz 84 model, where we experiment with the influence of dierent observation or measurement operators on the update.

Cone penetration test (CPT) is widely used in geotechnical site investigation. Several methods have been developed to automatically identify soil stratigraphy based on a single CPT sounding, which is an essential step in interpreting CPT data. Among existing methods, Bayesian methods allow probabilistic reasoning of soil stratigraphy from a given CPT sounding with explicit uncertainty quantification. However, Bayesian soil stratification methods based on CPT data tend to be computationally demanding or even prohibitive, particularly at sites with excessive soil strata due to the high dimensionality of soil stratification models. To address this issue, this study proposes a Bayesian method for fast soil stratification based on a single CPT sounding using a trans-dimensional sampling technique known as reversible jump Markov chain Monte Carlo (RJMCMC) simulation. The RJMCMC simulation is specifically tailored for Bayesian soil stratification by developing proposal distributions for three different Markov chain moves. The proposed method is illustrated and validated using benchmark examples and two real-life CPT soundings. It accomplishes CPT-based probabilistic soil stratification within 1–2 min. The computational time is generally unaffected by CPT sounding depth or the number of soil layers, making it feasible to stratify soil profiles with an excessive number of soil layers.

<div>We present a framework to solve geophysical inverse problems using the Hamiltonian Monte Carlo (HMC) method, with a focus on Bayesian tomography. Recent work in the geophysical community has shown the potential for gradient-based Monte Carlo sampling for a wide range of inverse problems across several fields.</div><div> </div><div>Many high-dimensional (non-linear) problems in geophysics have readily accessible gradient information which is unused in classical probabilistic inversions. Using HMC is a way to help improve traditional Monte Carlo sampling while increasing the scalability of inference problems, allowing access to uncertainty quantification for problems with many free parameters (>10'000). The result of HMC sampling is a collection of models representing the posterior probability density function, from which not only "best" models can be inferred, but also uncertainties and potentially different plausible scenarios, all compatible with the observed data. However, the amount of tuning parameters required by HMC, as well as the complexity of existing statistical modeling software, has limited the geophysical community in widely adopting a specific tool for performing efficient large-scale Bayesian inference.</div><div> </div><div>This work attempts to make a step towards filling that gap by providing an HMC sampler tailored for geophysical inverse problems (by e.g. supplying relevant priors and visualizations) combined with a set of different forward models, ranging from elastic and acoustic wave propagation to magnetic anomaly modeling, traveltimes, etc.. The framework is coded in the didactic but performant languages Julia and Python, with the possibility for the user to combine their own forward models, which are linked to the sampler routines by proper interfaces. In this way, we hope to illustrate the usefulness and potential of HMC in Bayesian inference. Tutorials featuring an array of physical experiments are written with the aim of both showcasing Bayesian inference and successful HMC usage. It additionally includes examples on how to speed up HMC e.g. with automated tuning techniques and GPU computations.</div>

Batho correction algorithm has high correction accuracy in relatively homogeneity tissue, but has low accuracy in cavum region and the interface between different density tissues, which can't meet the accuracy requirement of clinical application. In this paper, the changes of dose distribution with every kinds of different density tissues combination are researched by Monte Carlo(MC) simulation, the database of dose distribution on non-uniform tissue is built up for the purpose of development fast and accurate dose calculation method.

In this paper, the shape and size parameters (without imposing either parameter) and the survival function of the inverse Lomax distribution in the presence of observational data of the second type are estimated using different Bayesian methods, which included the standard Bayesian method and the hierarchical Bayesian method under a symmetric loss function, which is the quadratic loss function, and in order to compare The two methods used Monte Carlo simulation, where different sample sizes and three models of default values for distribution parameters (Ѳ,λ) , were used and the statistical measure, mean integral square error (IMSE), was used. Simulation experiments showed the superiority of the hierarchical Bayesian method because it has the least mean square integral error (IMSE)

This paper outlines a logical process based on Bayesian inference to exploit monitoring observations to improve the accuracy in the reliability estimates of civil structures that are retrofitted during their service lives. The proposed approach is particularly effective for complex structures, whose response prediction is affected by structural parameters with significant uncertainty; typical examples are prestressed concrete bridges, heavily influenced by the inherent uncertainties of the material’s creep and shrinkage. The methodology is applied to a real-life complex bridge in Italy, as opposed to academic problems and laboratory tests; the structural model developed for this study includes the original creep law of the Model B3 proposed by Bazant rather than linear approximation or response surfaces. The Bayesian inference is performed using a Markov Chain Monte Carlo (MCMC) method, while the reliability assessment utilises importance sampling (IS) for computational efficiency. The results show the extent to which structural health monitoring (SHM) and Bayesian inference reduce the uncertainty in structural parameters and the prediction of the structural demand. At the same time, they increase structural reliability. Particular attention is given to computational effort, structural modelling and algorithm optimisation.

Considering the impact of the heterogeneous conditions of the mixture baseline distribution on the parameter estimation of a composite dynamical system (CDS), we propose an approach to infer the model parameters and baseline survival function of CDS using the maximum likelihood estimation and Bayesian estimation methods. The power-trend hazard rate function and Burr type XII mixture distribution as the baseline distribution are used to characterize the changes of the residual lifetime distribution of surviving components. The Markov chain Monte Carlo approach via using a new Metropolis–Hastings within the Gibbs sampling algorithm is proposed to overcome the computation complexity when obtaining the Bayes estimates of model parameters. A numerical example is generated from the proposed CDS to analyze the proposed procedure. Monte Carlo simulations are conducted to investigate the performance of the proposed methods, and results show that the proposed Bayesian estimation method outperforms the maximum likelihood estimation method to obtain reliable estimates of the model parameters and baseline survival function in terms of the bias and mean square error.

Considers some statistical aspects of inverse problems, using Bayesian analysis, particularly estimation and hypothesis-testing questions for parameter-dependent differential equations. The author relates Bayesian maximum likelihood to Tikhonov regularization and applies the expectation-minimization algorithm to the problem of setting regularization levels. Further, he compares Bayesian results with those of a classical statistical approach, through consistency and asymptotic normality. A numerical example illustrates the application of Bayesian techniques. In many cases one is interested in parameters which are infinite dimensional (e.g. functions). Bayesian techniques offer a sound theoretical and computational paradigm, through probability measures on Banach space. He develops a framework for infinite dimensional Bayesian analysis, including convergence of approximations required to perform inference tasks computationally.

In this study, we employ Bayesian statistical methods alongside deep learning techniques to construct Friedmann–Robertson–Walker (FRW) cosmological models within the framework of [Formula: see text] gravity. We specifically investigate the accelerating expansion of the universe by considering cosmological dynamics consistent with a cubic form of the [Formula: see text] gravity model. This is achieved through an innovative and analytically tractable parameterization of the Hubble parameter, expressed as [Formula: see text]. The free parameters [Formula: see text], [Formula: see text], and [Formula: see text] are constrained using Bayesian Markov Chain Monte Carlo (MCMC) methods. Our results demonstrate consistency with observational cosmology, affirming the viability of our model. Furthermore, we analyze the physical properties of the universe within this framework, examining key cosmological quantities such as energy density, pressure, and the equation of state parameter. We also explore kinematic diagnostics, including the Hubble parameter, deceleration parameter, and the Om(z) diagnostic, to gain deeper insights into cosmic evolution. To enhance parameter estimation accuracy, we introduce a novel approach leveraging a Mixed Neural Network (MNN), which integrates Artificial Neural Networks (ANNs) with Mixture Density Networks (MDN). By synergizing the strengths of these techniques, our method significantly improves the precision of cosmological parameter inference.

An accurate prediction of circuit delay distribution is essential to verify the timing closure of digital circuits and to estimate the parametric yield. In the presence of process variations, the most accurate technique for circuit delay prediction is using Monte Carlo simulations. However, to get an accurate estimate, a large number of Monte Carlo simulations are required which is infeasible in the case of large circuits. Although statistical timing analysis techniques are widely used to predict the circuit delay distribution in a reasonable run time, the standard deviation of circuit delay is often inaccurate. In this work, an efficient technique for circuit delay prediction using limited number of Monte Carlo simulations is proposed. This is done using the concept of Bayesian inference with the results from statistical timing analysis as the prior information. The results indicate that combining statistical timing analysis along with limited number of Monte Carlo simulations increases the accuracy of prediction of circuit delay variance. The number of Monte Carlo simulations can be decided based on the accuracy requirements or run time constraints.

Chapter 56 Computationally intensive methods for integration in econometrics