Comment on se-2021-156

The Hamiltonian Monte Carlo (HMC) algorithm is a Markov Chain Monte Carlo (MCMC) technique, which combines the advantages of Hamiltonian dynamics methods and Metropolis Monte Carlo approach, to sample from complex distributions. The HMC algorithm incorporates gradient information in the dynamic trajectories and thus suppresses the random walk nature in traditional Markov chain simulation methods. This ensures rapid mixing, faster convergence, and improved efficiency of the Markov chain. The leapfrog method is generally used in discrete simulation of the dynamic transitions. In this paper, we refer to this as the leapfrog–HMC. The primary goal of this paper is to present the HMC algorithm as a tool for rapid sampling of high dimensional and complex distributions, and demonstrate its advantages over the classical Metropolis Monte Carlo technique. We demonstrate that the use of an adaptive–step discretization scheme in simulating the dynamic transitions results in an algorithm which significantly outperforms the leapfrog–HMC algorithm. Relevance to reservoir parameter estimation and uncertainty quantification will be discussed.

A reliable assessment of the posterior uncertainties is a crucial aspect of any amplitude versus angle (AVA) inversion due to the severe ill-conditioning of this inverse problem. To accomplish this task, numerical Markov chain Monte Carlo algorithms are usually used when the forward operator is nonlinear. The downside of these algorithms is the considerable number of samples needed to attain stable posterior estimations especially in high-dimensional spaces. To overcome this issue, we assessed the suitability of Hamiltonian Monte Carlo (HMC) algorithm for nonlinear target- and interval-oriented AVA inversions for the estimation of elastic properties and associated uncertainties from prestack seismic data. The target-oriented approach inverts the AVA responses of the target reflection by adopting the nonlinear Zoeppritz equations, whereas the interval-oriented method inverts the seismic amplitudes along a time interval using a 1D convolutional forward model still based on the Zoeppritz equations. HMC uses an artificial Hamiltonian system in which a model is viewed as a particle moving along a trajectory in an extended space. In this context, the inclusion of the derivative information of the misfit function makes possible long-distance moves with a high probability of acceptance from the current position toward a new independent model. In our application, we adopt a simple Gaussian a priori distribution that allows for an analytical inclusion of geostatistical constraints into the inversion framework, and we also develop a strategy that replaces the numerical computation of the Jacobian with a matrix operator analytically derived from a linearization of the Zoeppritz equations. Synthetic and field data inversions demonstrate that the HMC is a very promising approach for Bayesian AVA inversion that guarantees an efficient sampling of the model space and retrieves reliable estimations and accurate uncertainty quantifications with an affordable computational cost.

We explore a general framework in Markov chain Monte Carlo (MCMC) sampling where sequential proposals are tried as a candidate for the next state of the Markov chain. This sequential-proposal framework can be applied to various existing MCMC methods, including Metropolis–Hastings algorithms using random proposals and methods that use deterministic proposals such as Hamiltonian Monte Carlo (HMC) or the bouncy particle sampler. Sequential-proposal MCMC methods construct the same Markov chains as those constructed by the delayed rejection method under certain circumstances. In the context of HMC, the sequential-proposal approach has been proposed as extra chance generalized hybrid Monte Carlo (XCGHMC). We develop two novel methods in which the trajectories leading to proposals in HMC are automatically tuned to avoid doubling back, as in the No-U-Turn sampler (NUTS). The numerical efficiency of these new methods compare favorably to the NUTS. We additionally show that the sequential-proposal bouncy particle sampler enables the constructed Markov chain to pass through regions of low target density and thus facilitates better mixing of the chain when the target density is multimodal.

The aim of this work is to tackle the problem of sampling from multi-modal distributions when Hybrid Monte Carlo (HMC) algorithm is employed for performing Bayesian inference in dynamical systems. Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo (MCMC) algorithm but it still suffers from the “multiple peaks” problem. Due to non-trivial structure in the space of (a class of) dynamical systems, posterior distribution of its model parameters could exhibit complicated structures such as multiple ridges. We examined a MCMC algorithm combining HMC with so-called Parallel Tempering (PT) - a well-known strategy for tackling the problem highlighted above. The new algorithm is referred to as PT-HMC. Our numerical experiment demonstrated that when compared to the ground truth, the posterior distributions derived from PT-HMC samples is more accurate than those from HMC.

This article reviews the Bayesian inference with the Monte Carlo Markov Chain (MCMC) and the Hamiltonian Monte Carlo (HMC) samplers as a competitor of the classical likelihood statistical inference for pharmacometricians. The MCMC and the HMC samplers have greatly contributed to realization of the Bayesian methods with minimal requirement of mathematical theory. They do not require any closed form of the posterior density nor linear approximation of complex nonlinear models in high dimension even with non-conjugate priors. The HMC even weakens the dependency of the chain and improves computational efficiency. Pharmacometrics is one of great beneficiaries since they use complex multivariate multilevel nonlinear mixed effects models based on the restricted maximum likelihood estimation. Comprehension of the Bayesian approach will help pharmacometricians to access the data analysis more conveniently.

Timely detection of localised COVID-19 surges is essential for targeting limited health resources, yet most routine surveillance algorithms ignore spatial dependence and many Bayesian spatio-temporal models are computationally demanding. Evidence on Hamiltonian Monte Carlo (HMC) performance for outbreak detection in low- and middle-income country (LMIC) settings remains limited. We applied a Bayesian spatio-temporal hidden Markov model (HMM) to South African COVID-19 hospital mortality, comparing data-augmented Markov chain Monte Carlo (MCMC) with dynamic HMC. We conducted a retrospective ecological time-series study of in-hospital COVID- 19 deaths across 52 districts over 28 months (March 2020-June 2022), using national hospital surveillance linked to district-level health-system and population indicators. Death counts were modelled with a Poisson log-linear specification incorporating a smooth temporal trend, cyclic seasonality, spatial smoothing, and selected covariates, offset by expected deaths from admissions and a case-fatality ratio. Outbreaks were represented by a two-state HMM with latent indicators integrated out analytically and estimated via dynamic HMC in Stan. Eight candidate models were ranked using marginal likelihoods; the preferred model was re-fitted with both samplers to compare runtime, ESS, and convergence. A spatial HMM with marginalised outbreak states was strongly favoured over nonoutbreak and threshold-based alternatives. Posterior outbreak probabilities reproduced the four recognised national waves while revealing marked district-level heterogeneity, with early intense outbreaks in Western Cape and Gauteng districts and later peaks inland. Outbreaks were short-lived (mean around three months), and residual spatial risks indicated persistent excess mortality in the Eastern Cape and Free State. Dynamic HMC and MCMC yielded similar outbreak probability surfaces; however, HMC produced substantially larger ESS 1 (approximately 5,000 versus 63) and near-ideal convergence, whereas MCMC showed poor mixing. ESS per second was similar, so HMC's extra computation yielded more informative samples. A Bayesian spatio-temporal HMM fitted with dynamic HMC delivers coherent, spatially resolved outbreak probabilities and captures short-lived district-level mortality surges within broader national waves. Despite greater computational intensity, dynamic HMC offers superior mixing and convergence and is preferable for routine surveillance when adequate computing resources are available. The framework is transferable to other routinely collected surveillance data in South Africa and similar LMIC settings.

Hamiltonian Monte Carlo (HMC) is a powerful framework for sampling from high-dimensional continuous distributions. Langevin Monte Carlo (LMC) is a special case of HMC that is widely used in Deep Learning applications. Given an n-dimensional continuous density P(X), the only requirement for implementing HMC is the differentiability of the energy \(U(X) = - \log P(X)\). Like other MCMC methods (e.g. slice sampling, Swendsen-Wang cuts), HMC introduces auxiliary variables to facilitate movement in the original space. In HMC, the original variables represent position, and the auxiliary variables represent momentum. Each position dimension has a single corresponding momentum variable, so the joint space of the original and auxiliary variables has dimension 2n, twice the size of the original space. Once the momentum variables are introduced, Hamilton’s Equations are used to simulate the time evolution of a physical system with potential energy U. The properties of Hamilton’s Equations ensure that movement in the joint space preserves the distribution of P in the original space.

We obtain several quantitative bounds on the mixing properties of an “ideal” Hamiltonian Monte Carlo (HMC) Markov chain for a strongly log-concave target distribution π on Rd. Our main result says that the HMC Markov chain generates a sample with Wasserstein error ϵ in roughly O(κ2log(1ϵ)) steps, where the condition number κ=M2 m2 is the ratio of the maximum M2 and minimum m2 eigenvalues of the Hessian of −log(π). In particular, this mixing bound does not depend explicitly on the dimension d. These results significantly extend and improve previous quantitative bounds on the mixing of ideal HMC, and can be used to analyze more realistic HMC algorithms. The main ingredient of our argument is a proof that initially “parallel” Hamiltonian trajectories contract over much longer steps than would be predicted by previous heuristics based on the Jacobi manifold.

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.

This study addresses the challenge of estimating parameters for two logistic populations that share a common scale parameter but have different location parameters in the presence of fuzzy data. To handle these complexities, both Maximum Likelihood Estimation (MLE) and Bayesian methods are employed. Asymptotic confidence intervals are constructed using ML estimates. For Bayesian estimation, a conjugate prior is utilized, and Bayes estimators are approximated using Lindley’s method due to the lack of closed-form solutions. Furthermore, Approximate Bayesian Computation (ABC) and Markov Chain Monte Carlo (MCMC) techniques, including Hamiltonian Monte Carlo (HMC) and the Metropolis–Hastings (MH) algorithm, are utilized to sample from the posterior distributions and construct Highest Posterior Density (HPD) intervals. A detailed comparative analysis of MLE, Lindley’s approximation, ABC, HMC, and MH is conducted to assess their performance. The effectiveness of the proposed methodology is demonstrated using a real-world dataset under fuzzy conditions.

We apply the hybrid Monte Carlo (HMC) algorithm to the financial time sires analysis of the stochastic volatility (SV) model for the first time. The HMC algorithm is used for the Markov chain Monte Carlo (MCMC) update of volatility variables of the SV model in the Bayesian inference. We compute parameters of the SV model from the artificial financial data and compare the results from the HMC algorithm with those from the Metropolis algorithm. We find that the HMC decorrelates the volatility variables faster than the Metropolis algorithm. We also make an empirical analysis based on the Yen/Dollar exchange rates.

The Hamiltonian Monte Carlo (HMC) sampling algorithm exploits Hamiltonian dynamics to construct efficient Markov Chain Monte Carlo (MCMC), which has become increasingly popular in machine learning and statistics. Since HMC uses the gradient information of the target distribution, it can explore the state space much more efficiently than random-walk proposals, but may suffer from high autocorrelation. In this paper, we propose Langevin Hamiltonian Monte Carlo (LHMC) to reduce the autocorrelation of the samples. Probabilistic inference involving multi-modal distributions is very difficult for dynamics-based MCMC samplers, which is easily trapped in the mode far away from other modes. To tackle this issue, we further propose a variational hybrid Monte Carlo (VHMC) which uses a variational distribution to explore the phase space and find new modes, and it is capable of sampling from multi-modal distributions effectively. A formal proof is provided that shows that the proposed method can converge to target distributions. Both synthetic and real datasets are used to evaluate its properties and performance. The experimental results verify the theory and show superior performance in multi-modal sampling.

Among Markov chain Monte Carlo algorithms, Hamiltonian Monte Carlo (HMC) is often the algorithm of choice for complex, high-dimensional target distributions; however, its efficiency is notoriously sensitive to the choice of the integration-time tuning parameter. When integrating both forward and backward in time using the same leapfrog integration step as HMC, the set of apogees, local maxima in the potential along a path, is the same whatever point (position and momentum) along the path is chosen to initialize the integration. We present the Apogee to Apogee Path Sampler (AAPS), which uses this invariance to create a simple yet generic methodology for constructing a path, proposing a point from it and accepting or rejecting that proposal so as to target the intended distribution. We demonstrate empirically that AAPS has a similar efficiency to HMC but is much more robust to the setting of its equivalent tuning parameter, the number of apogees that the path crosses. Supplementary materials for this article are available online.

Bayesian model updating provides a rigorous framework to account for uncertainty induced by lack of knowledge about engineering systems in their respective mathematical models through updates of the joint probability density function (PDF), the so-called posterior PDF, of the unknown model parameters. The Markov chain Monte Carlo (MCMC) methods are currently the most popular approaches for generating samples from the posterior PDF. However, these methods often found wanting when sampling from difficult distributions (e.g., high-dimensional PDFs, PDFs with flat manifolds, multimodal PDFs, and very peaked PDFs). This paper introduces a new multi-level sampling approach for Bayesian model updating, called Sequential Gauss-Newton algorithm, which is inspired by the Transitional Markov chain Monte Carlo (TMCMC) algorithm. The Sequential Gauss-Newton algorithm improves two aspects of TMCMC to make an efficient and effective MCMC algorithm for drawing samples from difficult posterior PDFs. First, the statistical efficiency of the algorithm is enhanced by use of the systematic resampling scheme. Second, a new MCMC algorithm, called Gauss-Newton MCMC algorithm, is proposed which is essentially an M-H algorithm with a Gaussian proposal PDF tailored to the posterior PDF using the gradient and Hessian information of the negative log posterior. The effectiveness of the proposed algorithm for solving the Bayesian model updating problem is illustrated using three examples with irregularly shaped posterior PDFs.

One of the open challenges in quantum computing is to find meaningful and practical methods to leverage quantum computation to accelerate classical machine-learning workflows. A ubiquitous problem in machine-learning workflows is sampling from probability distributions that we only have access to via their log probability. To this end, we extend the well-known Hamiltonian Monte Carlo (HMC) method for Markov chain Monte Carlo (MCMC) sampling to leverage quantum computation in a hybrid manner as a proposal function. Our new algorithm, Quantum Dynamical Hamiltonian Monte Carlo (QD-HMC), replaces the classical symplectic integration proposal step with simulations of quantum-coherent continuous-space dynamics on digital or analog quantum computers. We show that QD-HMC maintains key characteristics of HMC, such as maintaining the detailed balanced condition with momentum inversion, while also having the potential for polynomial speedups over its classical counterpart in certain scenarios. As sampling is a core subroutine in many forms of probabilistic inference, and MCMC in continuously parametrized spaces covers a large class of potential applications, this work widens the areas of applicability of quantum devices. Published by the American Physical Society 2024