The Hamiltonian Monte Carlo Algorithm in Parameter Estimation and Uncertainty Quantification

<strong class="journal-contentHeaderColor">Abstract.</strong> We present an efficient probabilistic workflow for the estimation of source parameters of induced seismic events in three-dimensional heterogeneous media. Our workflow exploits a linearized variant of the Hamiltonian Monte Carlo (HMC) algorithm. Compared to traditional Markov-Chain Monte Carlo (MCMC) algorithms, HMC is highly efficient in sampling high-dimensional model spaces. Through a linearization of the forward problem around the prior mean (i.e., the "best" initial model), this efficiency can be further improved. We show, however, that this linearization leads to a performance in which the output of an HMC chain strongly depends on the quality of the prior; in particular, because not all (induced) earthquake model parameters have a linear relationship with the recordings observed at the surface. To mitigate the importance of an accurate prior, we integrate the linearized HMC scheme into a workflow that (i) allows for a weak prior through linearization around various (initial) centroid locations, (ii) is able to converge to the mode containing the model with the (global) minimum misfit by means of an iterative HMC approach, and (iii) uses variance reduction as a criterion to include the output of individual Markov chains in the estimation of the posterior probability. Using a three-dimensional heterogeneous subsurface model of the Groningen gas field, we simulate an induced earthquake to test our workflow. We then demonstrate the virtue of our workflow by estimating the event's centroid (three parameters), moment tensor (six parameters), and the earthquake's origin time. We find that our workflow is able to recover the posterior probability of these source parameters rather well, even when the prior model information is inaccurate, imprecise, or both inaccurate and imprecise.

A reliable assessment of the posterior uncertainties is a crucial aspect of any amplitude versus angle (AVA) inversion due to the severe ill-conditioning of this inverse problem. To accomplish this task, numerical Markov chain Monte Carlo algorithms are usually used when the forward operator is nonlinear. The downside of these algorithms is the considerable number of samples needed to attain stable posterior estimations especially in high-dimensional spaces. To overcome this issue, we assessed the suitability of Hamiltonian Monte Carlo (HMC) algorithm for nonlinear target- and interval-oriented AVA inversions for the estimation of elastic properties and associated uncertainties from prestack seismic data. The target-oriented approach inverts the AVA responses of the target reflection by adopting the nonlinear Zoeppritz equations, whereas the interval-oriented method inverts the seismic amplitudes along a time interval using a 1D convolutional forward model still based on the Zoeppritz equations. HMC uses an artificial Hamiltonian system in which a model is viewed as a particle moving along a trajectory in an extended space. In this context, the inclusion of the derivative information of the misfit function makes possible long-distance moves with a high probability of acceptance from the current position toward a new independent model. In our application, we adopt a simple Gaussian a priori distribution that allows for an analytical inclusion of geostatistical constraints into the inversion framework, and we also develop a strategy that replaces the numerical computation of the Jacobian with a matrix operator analytically derived from a linearization of the Zoeppritz equations. Synthetic and field data inversions demonstrate that the HMC is a very promising approach for Bayesian AVA inversion that guarantees an efficient sampling of the model space and retrieves reliable estimations and accurate uncertainty quantifications with an affordable computational cost.

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.

We obtain several quantitative bounds on the mixing properties of an “ideal” Hamiltonian Monte Carlo (HMC) Markov chain for a strongly log-concave target distribution π on Rd. Our main result says that the HMC Markov chain generates a sample with Wasserstein error ϵ in roughly O(κ2log(1ϵ)) steps, where the condition number κ=M2 m2 is the ratio of the maximum M2 and minimum m2 eigenvalues of the Hessian of −log(π). In particular, this mixing bound does not depend explicitly on the dimension d. These results significantly extend and improve previous quantitative bounds on the mixing of ideal HMC, and can be used to analyze more realistic HMC algorithms. The main ingredient of our argument is a proof that initially “parallel” Hamiltonian trajectories contract over much longer steps than would be predicted by previous heuristics based on the Jacobi manifold.

Hamiltonian Monte Carlo (HMC) is a powerful framework for sampling from high-dimensional continuous distributions. Langevin Monte Carlo (LMC) is a special case of HMC that is widely used in Deep Learning applications. Given an n-dimensional continuous density P(X), the only requirement for implementing HMC is the differentiability of the energy \(U(X) = - \log P(X)\). Like other MCMC methods (e.g. slice sampling, Swendsen-Wang cuts), HMC introduces auxiliary variables to facilitate movement in the original space. In HMC, the original variables represent position, and the auxiliary variables represent momentum. Each position dimension has a single corresponding momentum variable, so the joint space of the original and auxiliary variables has dimension 2n, twice the size of the original space. Once the momentum variables are introduced, Hamilton’s Equations are used to simulate the time evolution of a physical system with potential energy U. The properties of Hamilton’s Equations ensure that movement in the joint space preserves the distribution of P in the original space.

This study addresses the challenge of estimating parameters for two logistic populations that share a common scale parameter but have different location parameters in the presence of fuzzy data. To handle these complexities, both Maximum Likelihood Estimation (MLE) and Bayesian methods are employed. Asymptotic confidence intervals are constructed using ML estimates. For Bayesian estimation, a conjugate prior is utilized, and Bayes estimators are approximated using Lindley’s method due to the lack of closed-form solutions. Furthermore, Approximate Bayesian Computation (ABC) and Markov Chain Monte Carlo (MCMC) techniques, including Hamiltonian Monte Carlo (HMC) and the Metropolis–Hastings (MH) algorithm, are utilized to sample from the posterior distributions and construct Highest Posterior Density (HPD) intervals. A detailed comparative analysis of MLE, Lindley’s approximation, ABC, HMC, and MH is conducted to assess their performance. The effectiveness of the proposed methodology is demonstrated using a real-world dataset under fuzzy conditions.

This article reviews the Bayesian inference with the Monte Carlo Markov Chain (MCMC) and the Hamiltonian Monte Carlo (HMC) samplers as a competitor of the classical likelihood statistical inference for pharmacometricians. The MCMC and the HMC samplers have greatly contributed to realization of the Bayesian methods with minimal requirement of mathematical theory. They do not require any closed form of the posterior density nor linear approximation of complex nonlinear models in high dimension even with non-conjugate priors. The HMC even weakens the dependency of the chain and improves computational efficiency. Pharmacometrics is one of great beneficiaries since they use complex multivariate multilevel nonlinear mixed effects models based on the restricted maximum likelihood estimation. Comprehension of the Bayesian approach will help pharmacometricians to access the data analysis more conveniently.

Timely detection of localised COVID-19 surges is essential for targeting limited health resources, yet most routine surveillance algorithms ignore spatial dependence and many Bayesian spatio-temporal models are computationally demanding. Evidence on Hamiltonian Monte Carlo (HMC) performance for outbreak detection in low- and middle-income country (LMIC) settings remains limited. We applied a Bayesian spatio-temporal hidden Markov model (HMM) to South African COVID-19 hospital mortality, comparing data-augmented Markov chain Monte Carlo (MCMC) with dynamic HMC. We conducted a retrospective ecological time-series study of in-hospital COVID- 19 deaths across 52 districts over 28 months (March 2020-June 2022), using national hospital surveillance linked to district-level health-system and population indicators. Death counts were modelled with a Poisson log-linear specification incorporating a smooth temporal trend, cyclic seasonality, spatial smoothing, and selected covariates, offset by expected deaths from admissions and a case-fatality ratio. Outbreaks were represented by a two-state HMM with latent indicators integrated out analytically and estimated via dynamic HMC in Stan. Eight candidate models were ranked using marginal likelihoods; the preferred model was re-fitted with both samplers to compare runtime, ESS, and convergence. A spatial HMM with marginalised outbreak states was strongly favoured over nonoutbreak and threshold-based alternatives. Posterior outbreak probabilities reproduced the four recognised national waves while revealing marked district-level heterogeneity, with early intense outbreaks in Western Cape and Gauteng districts and later peaks inland. Outbreaks were short-lived (mean around three months), and residual spatial risks indicated persistent excess mortality in the Eastern Cape and Free State. Dynamic HMC and MCMC yielded similar outbreak probability surfaces; however, HMC produced substantially larger ESS 1 (approximately 5,000 versus 63) and near-ideal convergence, whereas MCMC showed poor mixing. ESS per second was similar, so HMC's extra computation yielded more informative samples. A Bayesian spatio-temporal HMM fitted with dynamic HMC delivers coherent, spatially resolved outbreak probabilities and captures short-lived district-level mortality surges within broader national waves. Despite greater computational intensity, dynamic HMC offers superior mixing and convergence and is preferable for routine surveillance when adequate computing resources are available. The framework is transferable to other routinely collected surveillance data in South Africa and similar LMIC settings.

We explore a general framework in Markov chain Monte Carlo (MCMC) sampling where sequential proposals are tried as a candidate for the next state of the Markov chain. This sequential-proposal framework can be applied to various existing MCMC methods, including Metropolis–Hastings algorithms using random proposals and methods that use deterministic proposals such as Hamiltonian Monte Carlo (HMC) or the bouncy particle sampler. Sequential-proposal MCMC methods construct the same Markov chains as those constructed by the delayed rejection method under certain circumstances. In the context of HMC, the sequential-proposal approach has been proposed as extra chance generalized hybrid Monte Carlo (XCGHMC). We develop two novel methods in which the trajectories leading to proposals in HMC are automatically tuned to avoid doubling back, as in the No-U-Turn sampler (NUTS). The numerical efficiency of these new methods compare favorably to the NUTS. We additionally show that the sequential-proposal bouncy particle sampler enables the constructed Markov chain to pass through regions of low target density and thus facilitates better mixing of the chain when the target density is multimodal.

This paper summarizes the recent developments of hamiltorch,1 which is a PyTorch-based library for Hamiltonian Monte Carlo (HMC). HMC is a gradient-based Markov chain Monte Carlo technique that has favorable scaling properties to high-dimensional parameter spaces, leading to it often becoming the sampling method of choice when derivatives are available. HMC is therefore especially suited to performing Bayesian inference over neural networks (NN). As a result hamiltorch has been specifically developed to sample from PyTorch NN models. The development of hamiltorch has also led to new sampling schemes that are part of the library. In this paper I will highlight such novelties as well as provide a brief introduction to the features of hamiltorch and highlight its use-cases in recently referenced work.

&amp;lt;div&amp;gt;We present a framework to solve geophysical inverse problems using the Hamiltonian Monte Carlo (HMC) method, with a focus on Bayesian tomography. Recent work in the geophysical community has shown the potential for gradient-based Monte Carlo sampling for a wide range of inverse problems across several fields.&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;amp;#160;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;Many high-dimensional (non-linear) problems in geophysics have readily accessible gradient information which is unused in classical probabilistic inversions. Using HMC is a way to help improve traditional Monte Carlo sampling while increasing the scalability of inference problems, allowing access to uncertainty quantification for problems with many free parameters (&amp;gt;10'000). The result of HMC sampling is a collection of models representing the posterior probability density function, from which not only &amp;quot;best&amp;quot; models can be inferred, but also uncertainties and potentially different plausible scenarios, all compatible with the observed data. However, the amount of tuning parameters required by HMC, as well as the complexity of existing statistical modeling software, has limited the geophysical community in widely adopting a specific tool for performing efficient large-scale Bayesian inference.&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;amp;#160;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;This work attempts to make a step towards filling that gap by providing an HMC sampler tailored for geophysical inverse problems (by e.g. supplying relevant priors and visualizations) combined with a set of different forward models, ranging from elastic and acoustic wave propagation to magnetic anomaly modeling, traveltimes, etc.. The framework is coded in the didactic but performant languages Julia and Python, with the possibility for the user to combine their own forward models, which are linked to the sampler routines by proper interfaces. In this way, we hope to illustrate the usefulness and potential of HMC in Bayesian inference. Tutorials featuring an array of physical experiments are written with the aim of both showcasing Bayesian inference and successful HMC usage. It additionally includes examples on how to speed up HMC e.g. with automated tuning techniques and GPU computations.&amp;lt;/div&amp;gt;

Models of Stellar Population Synthesis (SPS) provide a predictive framework for the spectral energy distribution (SED) of a galaxy. SPS predictions can be computationally intensive, creating a bottleneck for attempts to infer the physical properties of large populations of individual galaxies from their SEDs and photometry; these computational challenges are especially daunting for near-future cosmology surveys that will measure the photometry of billions of galaxies. In this paper, we explore a range of computational techniques aimed at accelerating SPS predictions of galaxy photometry using the JAX library to target GPUs. We study a particularly advantageous approximation to the calculation of galaxy photometry that speeds up the computation by a factor of 50 relative to the exact calculation. We introduce a novel technique for incorporating burstiness into models of galaxy star formation history that captures very short-timescale fluctuations with negligible increase in computation time. We study the performance of Hamiltonian Monte Carlo (HMC) algorithms in which individual chains are parallelized across independent GPU threads, finding that our pipeline can carry out Bayesian inference at a rate of approximately 1,000 galaxy posteriors per minute on a single GPU. Our results provide an update to standard benchmarks in the literature on the computational demands of SPS inference; our publicly available code enables previously-impractical Bayesian analyses of large galaxy samples, and includes several standalone modules that could be adopted to speedup existing SPS pipelines.

The aim of this work is to tackle the problem of sampling from multi-modal distributions when Hybrid Monte Carlo (HMC) algorithm is employed for performing Bayesian inference in dynamical systems. Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo (MCMC) algorithm but it still suffers from the “multiple peaks” problem. Due to non-trivial structure in the space of (a class of) dynamical systems, posterior distribution of its model parameters could exhibit complicated structures such as multiple ridges. We examined a MCMC algorithm combining HMC with so-called Parallel Tempering (PT) - a well-known strategy for tackling the problem highlighted above. The new algorithm is referred to as PT-HMC. Our numerical experiment demonstrated that when compared to the ground truth, the posterior distributions derived from PT-HMC samples is more accurate than those from HMC.

In this work we provide performance guarantees for hypocoercive nonreversible Monte-Carlo Markov chain (MCMC) samplers X t X_t with invariant measure μ ∗ \mu _* ; our results apply in particular to the Langevin equation, Hamiltonian Monte-Carlo, and the bouncy particle and zig-zag samplers. Specifically, we establish a concentration inequality of Bernstein type for ergodic averages 1 T ∫ 0 T f ( X t ) d t \frac {1}{T} \int _0^T f(X_t)\, dt . As a consequence we provide two types of performance guarantees: (a) explicit nonasymptotic confidence intervals for ∫ f d μ ∗ \int f d\mu _* when using a finite time ergodic average with given initial condition μ \mu and (b) uncertainty quantification (UQ) bounds, expressed in terms of relative entropy rate, on the bias of ∫ f d μ ∗ \int f d\mu _* when using an alternative or approximate processes X ~ t \widetilde {X}_t . (Results in (b) generalize results of the authors [ Uncertainty quantification for Markov processes via variational principles and functional inequalities , SIAM/ASA J. Uncertain. Quantif. 8 (2020), no. 2, 539–572] for coercive dynamics.) The concentration inequality is proved by combining the approach via Feynman-Kac semigroups first noted by Wu [ A deviation inequality for non-reversible Markov processes , Ann. Inst. H. Poincaré Probab. Statist. 36 (2000), no. 4, 435–445] with the hypocoercive estimates of Dolbeault, Mouhot, and Schmeiser [ Hypocoercivity for kinetic equations with linear relaxation terms , C. R. Math. Acad. Sci. Paris 347 (2009), no. 9–10, 511–516] and [ Hypocoercivity for linear kinetic equations conserving mass , Trans. Amer. Math. Soc. 367 (2015), no. 6, 3807–3828] developed for the Langevin equation and generalized to partially deterministic Markov processes by Andrieu, Durmus, Nüsken, and Roussel [ Hypocoercivity of piecewise deterministic Markov process-Monte Carlo , Ann. Appl. Probab. 31 (2021), no. 5, 2478–2517].

We apply the hybrid Monte Carlo (HMC) algorithm to the financial time sires analysis of the stochastic volatility (SV) model for the first time. The HMC algorithm is used for the Markov chain Monte Carlo (MCMC) update of volatility variables of the SV model in the Bayesian inference. We compute parameters of the SV model from the artificial financial data and compare the results from the HMC algorithm with those from the Metropolis algorithm. We find that the HMC decorrelates the volatility variables faster than the Metropolis algorithm. We also make an empirical analysis based on the Yen/Dollar exchange rates.