Hamiltonian and Langevin Monte Carlo

The Hamiltonian Monte Carlo (HMC) algorithm is a Markov Chain Monte Carlo (MCMC) technique, which combines the advantages of Hamiltonian dynamics methods and Metropolis Monte Carlo approach, to sample from complex distributions. The HMC algorithm incorporates gradient information in the dynamic trajectories and thus suppresses the random walk nature in traditional Markov chain simulation methods. This ensures rapid mixing, faster convergence, and improved efficiency of the Markov chain. The leapfrog method is generally used in discrete simulation of the dynamic transitions. In this paper, we refer to this as the leapfrog–HMC. The primary goal of this paper is to present the HMC algorithm as a tool for rapid sampling of high dimensional and complex distributions, and demonstrate its advantages over the classical Metropolis Monte Carlo technique. We demonstrate that the use of an adaptive–step discretization scheme in simulating the dynamic transitions results in an algorithm which significantly outperforms the leapfrog–HMC algorithm. Relevance to reservoir parameter estimation and uncertainty quantification will be discussed.

This article reviews the Bayesian inference with the Monte Carlo Markov Chain (MCMC) and the Hamiltonian Monte Carlo (HMC) samplers as a competitor of the classical likelihood statistical inference for pharmacometricians. The MCMC and the HMC samplers have greatly contributed to realization of the Bayesian methods with minimal requirement of mathematical theory. They do not require any closed form of the posterior density nor linear approximation of complex nonlinear models in high dimension even with non-conjugate priors. The HMC even weakens the dependency of the chain and improves computational efficiency. Pharmacometrics is one of great beneficiaries since they use complex multivariate multilevel nonlinear mixed effects models based on the restricted maximum likelihood estimation. Comprehension of the Bayesian approach will help pharmacometricians to access the data analysis more conveniently.

Timely detection of localised COVID-19 surges is essential for targeting limited health resources, yet most routine surveillance algorithms ignore spatial dependence and many Bayesian spatio-temporal models are computationally demanding. Evidence on Hamiltonian Monte Carlo (HMC) performance for outbreak detection in low- and middle-income country (LMIC) settings remains limited. We applied a Bayesian spatio-temporal hidden Markov model (HMM) to South African COVID-19 hospital mortality, comparing data-augmented Markov chain Monte Carlo (MCMC) with dynamic HMC. We conducted a retrospective ecological time-series study of in-hospital COVID- 19 deaths across 52 districts over 28 months (March 2020-June 2022), using national hospital surveillance linked to district-level health-system and population indicators. Death counts were modelled with a Poisson log-linear specification incorporating a smooth temporal trend, cyclic seasonality, spatial smoothing, and selected covariates, offset by expected deaths from admissions and a case-fatality ratio. Outbreaks were represented by a two-state HMM with latent indicators integrated out analytically and estimated via dynamic HMC in Stan. Eight candidate models were ranked using marginal likelihoods; the preferred model was re-fitted with both samplers to compare runtime, ESS, and convergence. A spatial HMM with marginalised outbreak states was strongly favoured over nonoutbreak and threshold-based alternatives. Posterior outbreak probabilities reproduced the four recognised national waves while revealing marked district-level heterogeneity, with early intense outbreaks in Western Cape and Gauteng districts and later peaks inland. Outbreaks were short-lived (mean around three months), and residual spatial risks indicated persistent excess mortality in the Eastern Cape and Free State. Dynamic HMC and MCMC yielded similar outbreak probability surfaces; however, HMC produced substantially larger ESS 1 (approximately 5,000 versus 63) and near-ideal convergence, whereas MCMC showed poor mixing. ESS per second was similar, so HMC's extra computation yielded more informative samples. A Bayesian spatio-temporal HMM fitted with dynamic HMC delivers coherent, spatially resolved outbreak probabilities and captures short-lived district-level mortality surges within broader national waves. Despite greater computational intensity, dynamic HMC offers superior mixing and convergence and is preferable for routine surveillance when adequate computing resources are available. The framework is transferable to other routinely collected surveillance data in South Africa and similar LMIC settings.

<strong class="journal-contentHeaderColor">Abstract.</strong> We present an efficient probabilistic workflow for the estimation of source parameters of induced seismic events in three-dimensional heterogeneous media. Our workflow exploits a linearized variant of the Hamiltonian Monte Carlo (HMC) algorithm. Compared to traditional Markov-Chain Monte Carlo (MCMC) algorithms, HMC is highly efficient in sampling high-dimensional model spaces. Through a linearization of the forward problem around the prior mean (i.e., the "best" initial model), this efficiency can be further improved. We show, however, that this linearization leads to a performance in which the output of an HMC chain strongly depends on the quality of the prior; in particular, because not all (induced) earthquake model parameters have a linear relationship with the recordings observed at the surface. To mitigate the importance of an accurate prior, we integrate the linearized HMC scheme into a workflow that (i) allows for a weak prior through linearization around various (initial) centroid locations, (ii) is able to converge to the mode containing the model with the (global) minimum misfit by means of an iterative HMC approach, and (iii) uses variance reduction as a criterion to include the output of individual Markov chains in the estimation of the posterior probability. Using a three-dimensional heterogeneous subsurface model of the Groningen gas field, we simulate an induced earthquake to test our workflow. We then demonstrate the virtue of our workflow by estimating the event's centroid (three parameters), moment tensor (six parameters), and the earthquake's origin time. We find that our workflow is able to recover the posterior probability of these source parameters rather well, even when the prior model information is inaccurate, imprecise, or both inaccurate and imprecise.

We obtain several quantitative bounds on the mixing properties of an “ideal” Hamiltonian Monte Carlo (HMC) Markov chain for a strongly log-concave target distribution π on Rd. Our main result says that the HMC Markov chain generates a sample with Wasserstein error ϵ in roughly O(κ2log(1ϵ)) steps, where the condition number κ=M2 m2 is the ratio of the maximum M2 and minimum m2 eigenvalues of the Hessian of −log(π). In particular, this mixing bound does not depend explicitly on the dimension d. These results significantly extend and improve previous quantitative bounds on the mixing of ideal HMC, and can be used to analyze more realistic HMC algorithms. The main ingredient of our argument is a proof that initially “parallel” Hamiltonian trajectories contract over much longer steps than would be predicted by previous heuristics based on the Jacobi manifold.

We explore a general framework in Markov chain Monte Carlo (MCMC) sampling where sequential proposals are tried as a candidate for the next state of the Markov chain. This sequential-proposal framework can be applied to various existing MCMC methods, including Metropolis–Hastings algorithms using random proposals and methods that use deterministic proposals such as Hamiltonian Monte Carlo (HMC) or the bouncy particle sampler. Sequential-proposal MCMC methods construct the same Markov chains as those constructed by the delayed rejection method under certain circumstances. In the context of HMC, the sequential-proposal approach has been proposed as extra chance generalized hybrid Monte Carlo (XCGHMC). We develop two novel methods in which the trajectories leading to proposals in HMC are automatically tuned to avoid doubling back, as in the No-U-Turn sampler (NUTS). The numerical efficiency of these new methods compare favorably to the NUTS. We additionally show that the sequential-proposal bouncy particle sampler enables the constructed Markov chain to pass through regions of low target density and thus facilitates better mixing of the chain when the target density is multimodal.

The aim of this work is to tackle the problem of sampling from multi-modal distributions when Hybrid Monte Carlo (HMC) algorithm is employed for performing Bayesian inference in dynamical systems. Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo (MCMC) algorithm but it still suffers from the “multiple peaks” problem. Due to non-trivial structure in the space of (a class of) dynamical systems, posterior distribution of its model parameters could exhibit complicated structures such as multiple ridges. We examined a MCMC algorithm combining HMC with so-called Parallel Tempering (PT) - a well-known strategy for tackling the problem highlighted above. The new algorithm is referred to as PT-HMC. Our numerical experiment demonstrated that when compared to the ground truth, the posterior distributions derived from PT-HMC samples is more accurate than those from HMC.

This study addresses the challenge of estimating parameters for two logistic populations that share a common scale parameter but have different location parameters in the presence of fuzzy data. To handle these complexities, both Maximum Likelihood Estimation (MLE) and Bayesian methods are employed. Asymptotic confidence intervals are constructed using ML estimates. For Bayesian estimation, a conjugate prior is utilized, and Bayes estimators are approximated using Lindley’s method due to the lack of closed-form solutions. Furthermore, Approximate Bayesian Computation (ABC) and Markov Chain Monte Carlo (MCMC) techniques, including Hamiltonian Monte Carlo (HMC) and the Metropolis–Hastings (MH) algorithm, are utilized to sample from the posterior distributions and construct Highest Posterior Density (HPD) intervals. A detailed comparative analysis of MLE, Lindley’s approximation, ABC, HMC, and MH is conducted to assess their performance. The effectiveness of the proposed methodology is demonstrated using a real-world dataset under fuzzy conditions.

We apply the hybrid Monte Carlo (HMC) algorithm to the financial time sires analysis of the stochastic volatility (SV) model for the first time. The HMC algorithm is used for the Markov chain Monte Carlo (MCMC) update of volatility variables of the SV model in the Bayesian inference. We compute parameters of the SV model from the artificial financial data and compare the results from the HMC algorithm with those from the Metropolis algorithm. We find that the HMC decorrelates the volatility variables faster than the Metropolis algorithm. We also make an empirical analysis based on the Yen/Dollar exchange rates.

The Hamiltonian Monte Carlo (HMC) sampling algorithm exploits Hamiltonian dynamics to construct efficient Markov Chain Monte Carlo (MCMC), which has become increasingly popular in machine learning and statistics. Since HMC uses the gradient information of the target distribution, it can explore the state space much more efficiently than random-walk proposals, but may suffer from high autocorrelation. In this paper, we propose Langevin Hamiltonian Monte Carlo (LHMC) to reduce the autocorrelation of the samples. Probabilistic inference involving multi-modal distributions is very difficult for dynamics-based MCMC samplers, which is easily trapped in the mode far away from other modes. To tackle this issue, we further propose a variational hybrid Monte Carlo (VHMC) which uses a variational distribution to explore the phase space and find new modes, and it is capable of sampling from multi-modal distributions effectively. A formal proof is provided that shows that the proposed method can converge to target distributions. Both synthetic and real datasets are used to evaluate its properties and performance. The experimental results verify the theory and show superior performance in multi-modal sampling.

In Bayesian decision theory, stochastic simulation techniques have been widely used by decision makers. The Markov chain Monte Carlo method (MCMC) plays a key role in stochastic simulation within the framework of Bayesian statistics. The notion of "slice sampling", or employing auxiliary variables in sampling, has been recently suggested in the literature for improving the efficiency of the traditional MCMC methods. In the existing literature, the one dimensional slice sampler has been extensively studied, yet the literature on the multidimensional case is sparse. In this study, we utilize multiple auxiliary variables in our sampling algorithms for multivariate normal distributions, which adapt better to the local properties of the target probability distributions. We show that these methods are flexible enough to allow for truncation to rectangular regions and/or the exclusion of any n dimensional hyper-quadrant. We compare these algorithms for both efficiency and accuracy via simulation experiments. Our results show that our new sampling techniques are accurate and more effective than the traditional single auxiliary variable slice sampler, especially for truncated multivariate distributions. Further, we extend our methods to general multivariate distributions including multivariate student t distributions, multivariate elliptical distributions, multivariate skew normal distributions, multivariate skew t distributions and a general class of multivariate skew elliptical distributions. We also discuss and outline some applications of our algorithms in the real business world, especially in the production and operations management and the finance areas. With regard to production and operations management, we find that the proposed multivariate normal slice sampler can be implemented in a stochastic optimization process for finding the optimal fulfillment rate of an assembly system. This system consists of n components, from which m products are assembled via a periodic review control policy. In finance area, we show that multivariate slice samplers can be used to update model parameters and predict future asset returns in a higher moments Bayesian portfolio optimization model, which is based on a general class of multivariate skew t distributions. The application of a multivariate slice sampler improves the model's ability of handling large data sets and saves computation time for deriving complicated posterior probability distributions.

One of the open challenges in quantum computing is to find meaningful and practical methods to leverage quantum computation to accelerate classical machine-learning workflows. A ubiquitous problem in machine-learning workflows is sampling from probability distributions that we only have access to via their log probability. To this end, we extend the well-known Hamiltonian Monte Carlo (HMC) method for Markov chain Monte Carlo (MCMC) sampling to leverage quantum computation in a hybrid manner as a proposal function. Our new algorithm, Quantum Dynamical Hamiltonian Monte Carlo (QD-HMC), replaces the classical symplectic integration proposal step with simulations of quantum-coherent continuous-space dynamics on digital or analog quantum computers. We show that QD-HMC maintains key characteristics of HMC, such as maintaining the detailed balanced condition with momentum inversion, while also having the potential for polynomial speedups over its classical counterpart in certain scenarios. As sampling is a core subroutine in many forms of probabilistic inference, and MCMC in continuously parametrized spaces covers a large class of potential applications, this work widens the areas of applicability of quantum devices. Published by the American Physical Society 2024

Randomized physics-informed machine learning for uncertainty quantification in high-dimensional inverse problems

Among Markov chain Monte Carlo algorithms, Hamiltonian Monte Carlo (HMC) is often the algorithm of choice for complex, high-dimensional target distributions; however, its efficiency is notoriously sensitive to the choice of the integration-time tuning parameter. When integrating both forward and backward in time using the same leapfrog integration step as HMC, the set of apogees, local maxima in the potential along a path, is the same whatever point (position and momentum) along the path is chosen to initialize the integration. We present the Apogee to Apogee Path Sampler (AAPS), which uses this invariance to create a simple yet generic methodology for constructing a path, proposing a point from it and accepting or rejecting that proposal so as to target the intended distribution. We demonstrate empirically that AAPS has a similar efficiency to HMC but is much more robust to the setting of its equivalent tuning parameter, the number of apogees that the path crosses. Supplementary materials for this article are available online.

Crash severity of a hazardous material (HAZMAT) transporting truck increases manyfold compared with normal truck crash because of the possible exposure to dangerous substances. Crashes which involve a HAZMAT truck might result in a catastrophic incident causing horrendous damage to individuals involved in the crash. In-transit HAZMAT crashes in Wyoming caused a total damage of $3.1 million from 2015 to 2018. HAZMAT crashes on interstate roads represented 22% of the total HAZMAT crashes causing a total damage of $2.2 million, representing 71% of the cost of total damage. Previous studies in Wyoming investigated all vehicle crashes, including large truck crashes, but none has analyzed HAZMAT-related crashes or accounted for its type as a contributing factor. This study fills the gap by analyzing crash injury severity of HAZMAT-related crashes on all interstate freeways in Wyoming. Furthermore, the study introduces the No-U-Turn (NUT) Hamiltonian Monte Carlo (HMC) method of hierarchical Bayesian analysis into HAZMAT crash injury severity analysis. In recent developments, NUT HMC has been proven to be the most efficient Markov Chain Monte Carlo (MCMC) sampling method. The results showed that 30% of the unobserved heterogeneity arises from variation in summer and winter crashes which justifies the use of hierarchical model. Among the other covariates investigated, the population-averaged effects showed that number of trucks involved, hit-and-run crashes, animal-vehicle crashes, work-zone-related crashes, collision type, percentage of females involved, drivers’ drug/alcohol use, seat-belt use, crash location, roadway curves, and surface conditions significantly impact HAZMAT crash injury severity.