Dirichlet‐multinomial modelling outperforms alternatives for analysis of microbiome and other ecological count data

The Hamiltonian Monte Carlo (HMC) algorithm is a Markov Chain Monte Carlo (MCMC) technique, which combines the advantages of Hamiltonian dynamics methods and Metropolis Monte Carlo approach, to sample from complex distributions. The HMC algorithm incorporates gradient information in the dynamic trajectories and thus suppresses the random walk nature in traditional Markov chain simulation methods. This ensures rapid mixing, faster convergence, and improved efficiency of the Markov chain. The leapfrog method is generally used in discrete simulation of the dynamic transitions. In this paper, we refer to this as the leapfrog–HMC. The primary goal of this paper is to present the HMC algorithm as a tool for rapid sampling of high dimensional and complex distributions, and demonstrate its advantages over the classical Metropolis Monte Carlo technique. We demonstrate that the use of an adaptive–step discretization scheme in simulating the dynamic transitions results in an algorithm which significantly outperforms the leapfrog–HMC algorithm. Relevance to reservoir parameter estimation and uncertainty quantification will be discussed.

The Poisson regression model is widely used for count data. This model assumes equidispersion. In practice, equidispersion is seldom reflected in data. However, in real-life data, the variance usually exceeds the mean. This situation is known as overdispersion. Negative binomial distribution and other Poisson mix models are often used to model overdispersion count data. Another extension of the negative binomial distribution in another model for count data is the univariate generalized Waring. In addition, the model developed by Famoye can be used in the analysis of count data. When the count data contains a large number of zeros, it is necessary to use zero-inflated models. In this study, different generalized regression models are emphasized for the analysis of excessive zeros count data. For this purpose, a real data set was analysed with the generalized Poisson model, generalized negative binomial model, generalized negative binomial Famoye, generalized Waring model, and the foregoing zero-inflated models. Log-likelihood, Akaike information criterion, Bayes information criterion, Vuong statistics were used for model comparisons.

This article reviews the Bayesian inference with the Monte Carlo Markov Chain (MCMC) and the Hamiltonian Monte Carlo (HMC) samplers as a competitor of the classical likelihood statistical inference for pharmacometricians. The MCMC and the HMC samplers have greatly contributed to realization of the Bayesian methods with minimal requirement of mathematical theory. They do not require any closed form of the posterior density nor linear approximation of complex nonlinear models in high dimension even with non-conjugate priors. The HMC even weakens the dependency of the chain and improves computational efficiency. Pharmacometrics is one of great beneficiaries since they use complex multivariate multilevel nonlinear mixed effects models based on the restricted maximum likelihood estimation. Comprehension of the Bayesian approach will help pharmacometricians to access the data analysis more conveniently.

Timely detection of localised COVID-19 surges is essential for targeting limited health resources, yet most routine surveillance algorithms ignore spatial dependence and many Bayesian spatio-temporal models are computationally demanding. Evidence on Hamiltonian Monte Carlo (HMC) performance for outbreak detection in low- and middle-income country (LMIC) settings remains limited. We applied a Bayesian spatio-temporal hidden Markov model (HMM) to South African COVID-19 hospital mortality, comparing data-augmented Markov chain Monte Carlo (MCMC) with dynamic HMC. We conducted a retrospective ecological time-series study of in-hospital COVID- 19 deaths across 52 districts over 28 months (March 2020-June 2022), using national hospital surveillance linked to district-level health-system and population indicators. Death counts were modelled with a Poisson log-linear specification incorporating a smooth temporal trend, cyclic seasonality, spatial smoothing, and selected covariates, offset by expected deaths from admissions and a case-fatality ratio. Outbreaks were represented by a two-state HMM with latent indicators integrated out analytically and estimated via dynamic HMC in Stan. Eight candidate models were ranked using marginal likelihoods; the preferred model was re-fitted with both samplers to compare runtime, ESS, and convergence. A spatial HMM with marginalised outbreak states was strongly favoured over nonoutbreak and threshold-based alternatives. Posterior outbreak probabilities reproduced the four recognised national waves while revealing marked district-level heterogeneity, with early intense outbreaks in Western Cape and Gauteng districts and later peaks inland. Outbreaks were short-lived (mean around three months), and residual spatial risks indicated persistent excess mortality in the Eastern Cape and Free State. Dynamic HMC and MCMC yielded similar outbreak probability surfaces; however, HMC produced substantially larger ESS 1 (approximately 5,000 versus 63) and near-ideal convergence, whereas MCMC showed poor mixing. ESS per second was similar, so HMC's extra computation yielded more informative samples. A Bayesian spatio-temporal HMM fitted with dynamic HMC delivers coherent, spatially resolved outbreak probabilities and captures short-lived district-level mortality surges within broader national waves. Despite greater computational intensity, dynamic HMC offers superior mixing and convergence and is preferable for routine surveillance when adequate computing resources are available. The framework is transferable to other routinely collected surveillance data in South Africa and similar LMIC settings.

Overdispersion is common in models of count data in ecology and evolutionary biology, and can occur due to missing covariates, non-independent (aggregated) data, or an excess frequency of zeroes (zero-inflation). Accounting for overdispersion in such models is vital, as failing to do so can lead to biased parameter estimates, and false conclusions regarding hypotheses of interest. Observation-level random effects (OLRE), where each data point receives a unique level of a random effect that models the extra-Poisson variation present in the data, are commonly employed to cope with overdispersion in count data. However studies investigating the efficacy of observation-level random effects as a means to deal with overdispersion are scarce. Here I use simulations to show that in cases where overdispersion is caused by random extra-Poisson noise, or aggregation in the count data, observation-level random effects yield more accurate parameter estimates compared to when overdispersion is simply ignored. Conversely, OLRE fail to reduce bias in zero-inflated data, and in some cases increase bias at high levels of overdispersion. There was a positive relationship between the magnitude of overdispersion and the degree of bias in parameter estimates. Critically, the simulations reveal that failing to account for overdispersion in mixed models can erroneously inflate measures of explained variance (r2), which may lead to researchers overestimating the predictive power of variables of interest. This work suggests use of observation-level random effects provides a simple and robust means to account for overdispersion in count data, but also that their ability to minimise bias is not uniform across all types of overdispersion and must be applied judiciously.

Hamiltonian Monte Carlo (HMC) is a powerful framework for sampling from high-dimensional continuous distributions. Langevin Monte Carlo (LMC) is a special case of HMC that is widely used in Deep Learning applications. Given an n-dimensional continuous density P(X), the only requirement for implementing HMC is the differentiability of the energy \(U(X) = - \log P(X)\). Like other MCMC methods (e.g. slice sampling, Swendsen-Wang cuts), HMC introduces auxiliary variables to facilitate movement in the original space. In HMC, the original variables represent position, and the auxiliary variables represent momentum. Each position dimension has a single corresponding momentum variable, so the joint space of the original and auxiliary variables has dimension 2n, twice the size of the original space. Once the momentum variables are introduced, Hamilton’s Equations are used to simulate the time evolution of a physical system with potential energy U. The properties of Hamilton’s Equations ensure that movement in the joint space preserves the distribution of P in the original space.

Background: Data from 16S amplicon sequencing present challenges to ecological and statistical interpretation. In particular, library sizes often vary over several ranges of magnitude, and the data contains many zeroes. Also, since researchers sample a small fraction of the ecosystem, the observed sequences are relative abundances and therefore the data is compositional. Here we evaluate methods developed in the literature to address these three challenges in the context of normalization and ordination analysis, which is commonly used to visualize overall differences in bacterial composition between sample groups, and differential abundance analysis, which tests for significant differences in the abundances of microbes between sample groups. Results. Effects of normalization on ordination: Most normalization methods successfully cluster samples according to biological origin when many microbes differ between the groups. For datasets in which clusters are subtle and/or sequence depth varies greatly between samples, or for metrics in which rare microbes play an important role, rarefying outperforms other techniques. For abundance-based metrics, rarefying as well as alternatives like DESeq and metagenomeSeq’s cumulative sum scaling (CSS), seem to correctly cluster samples according to biological origin. With these normalization alternatives, clustering by sequence depth as a confounding variable must be checked for, especially for low library sizes. Effects of differential abundance testing model choice: We build on previous work to evaluate each statistical method using rarefied as well as unrarefied data. When the mean library sizes in the differential abundance groups differ by more than 2-3x, or the library sizes differ in distribution, our simulation studies reveal that each statistical method improved in its false positive rate when samples were rarefied. However, when the difference in library size mean is less than 2-3x, and the library sizes are similarly distributed, rarefying results in a loss of power for all methods. In this case, DESeq2 has the highest power to compare groups, especially for less than 20 samples per group. MetagenomeSeq’s fitZIG is a faster alternative to DESeq2, although it does worse for smaller sample sizes (<50 samples per group) and tends to have a higher false positives rate. For larger sample sizes (>50 samples), rarefying paired with a non-parametric test, such as the Mann-Whitney test, can also yield equally high sensitivity. Based on these results, we recommend a stepwise procedure in which sample groups are first tested for significant differences in library size. If there is a significant difference, we recommend rarefying with a non-parametric test. Otherwise, DESeq2 and/or fitZIG offer increased sensitivity, especially for rare OTUs and small sample numbers. Conclusions. These findings help guide which technique to use depending on the data characteristics of a given study.

<strong class="journal-contentHeaderColor">Abstract.</strong> We present an efficient probabilistic workflow for the estimation of source parameters of induced seismic events in three-dimensional heterogeneous media. Our workflow exploits a linearized variant of the Hamiltonian Monte Carlo (HMC) algorithm. Compared to traditional Markov-Chain Monte Carlo (MCMC) algorithms, HMC is highly efficient in sampling high-dimensional model spaces. Through a linearization of the forward problem around the prior mean (i.e., the "best" initial model), this efficiency can be further improved. We show, however, that this linearization leads to a performance in which the output of an HMC chain strongly depends on the quality of the prior; in particular, because not all (induced) earthquake model parameters have a linear relationship with the recordings observed at the surface. To mitigate the importance of an accurate prior, we integrate the linearized HMC scheme into a workflow that (i) allows for a weak prior through linearization around various (initial) centroid locations, (ii) is able to converge to the mode containing the model with the (global) minimum misfit by means of an iterative HMC approach, and (iii) uses variance reduction as a criterion to include the output of individual Markov chains in the estimation of the posterior probability. Using a three-dimensional heterogeneous subsurface model of the Groningen gas field, we simulate an induced earthquake to test our workflow. We then demonstrate the virtue of our workflow by estimating the event's centroid (three parameters), moment tensor (six parameters), and the earthquake's origin time. We find that our workflow is able to recover the posterior probability of these source parameters rather well, even when the prior model information is inaccurate, imprecise, or both inaccurate and imprecise.

Evaluating uncertainties in electrochemical impedance spectra of solid oxide fuel cells

In-line inspection (ILI) data is commonly used in corrosion growth models (CGMs) to predict the corrosion growth in energy pipelines. A hierarchical stochastic corrosion growth model is considered in this paper which considers the variations in the corrosion growth, both spatially and temporally, the inherent measurement error of the ILI tools as well as the model uncertainties. These uncertainties are represented as unknown model variables and are often inferred using a Bayesian method [1], [2] and samples of the unknown parameters’ posterior probability density functions (PDFs) are obtained using Markov Chain Monte Carlo (MCMC) sampling techniques [3]. ILIs can result in massive data sets. In order for MCMC-based inference techniques to yield reasonably accurate results, many samples (approaching infinity) are required. This fact in addition to the massive data sets exponentially increases the scale of the inference problem from an attainable solution to a potentially impossible one that is limited by today’s computing power. For this reason, MCMC-based inference techniques can become inefficient in the cases where ILI datasets are large. The objective is to propose variational inference (VI) as an alternative to MCMC to determine a Bayesian solution for the unknown parameters in complex stochastic CGMs. VI produces approximations of the posterior PDFs by treating the inference as an optimization problem. Variational inference emerged from machine learning for Bayesian inference of large data sets; therefore, it is an appropriate tool to use in the analysis of mass pipeline inspection data[4]–[7]. This paper introduces VI to solve the inference problem and provide a solution for a hierarchical stochastic CGM to describe the defect-specific corrosion growth experienced in pipelines based on excessively large ILI datasets. To gauge the accuracy of the VI implementation in the model, the results are compared to a set of values generated using a stochastic gamma process that represents the corrosion growth process experienced by the pipe.

We explore a general framework in Markov chain Monte Carlo (MCMC) sampling where sequential proposals are tried as a candidate for the next state of the Markov chain. This sequential-proposal framework can be applied to various existing MCMC methods, including Metropolis–Hastings algorithms using random proposals and methods that use deterministic proposals such as Hamiltonian Monte Carlo (HMC) or the bouncy particle sampler. Sequential-proposal MCMC methods construct the same Markov chains as those constructed by the delayed rejection method under certain circumstances. In the context of HMC, the sequential-proposal approach has been proposed as extra chance generalized hybrid Monte Carlo (XCGHMC). We develop two novel methods in which the trajectories leading to proposals in HMC are automatically tuned to avoid doubling back, as in the No-U-Turn sampler (NUTS). The numerical efficiency of these new methods compare favorably to the NUTS. We additionally show that the sequential-proposal bouncy particle sampler enables the constructed Markov chain to pass through regions of low target density and thus facilitates better mixing of the chain when the target density is multimodal.

Chapter 24 - Variational Bayes

We obtain several quantitative bounds on the mixing properties of an “ideal” Hamiltonian Monte Carlo (HMC) Markov chain for a strongly log-concave target distribution π on Rd. Our main result says that the HMC Markov chain generates a sample with Wasserstein error ϵ in roughly O(κ2log(1ϵ)) steps, where the condition number κ=M2 m2 is the ratio of the maximum M2 and minimum m2 eigenvalues of the Hessian of −log(π). In particular, this mixing bound does not depend explicitly on the dimension d. These results significantly extend and improve previous quantitative bounds on the mixing of ideal HMC, and can be used to analyze more realistic HMC algorithms. The main ingredient of our argument is a proof that initially “parallel” Hamiltonian trajectories contract over much longer steps than would be predicted by previous heuristics based on the Jacobi manifold.

This study addresses the challenge of estimating parameters for two logistic populations that share a common scale parameter but have different location parameters in the presence of fuzzy data. To handle these complexities, both Maximum Likelihood Estimation (MLE) and Bayesian methods are employed. Asymptotic confidence intervals are constructed using ML estimates. For Bayesian estimation, a conjugate prior is utilized, and Bayes estimators are approximated using Lindley’s method due to the lack of closed-form solutions. Furthermore, Approximate Bayesian Computation (ABC) and Markov Chain Monte Carlo (MCMC) techniques, including Hamiltonian Monte Carlo (HMC) and the Metropolis–Hastings (MH) algorithm, are utilized to sample from the posterior distributions and construct Highest Posterior Density (HPD) intervals. A detailed comparative analysis of MLE, Lindley’s approximation, ABC, HMC, and MH is conducted to assess their performance. The effectiveness of the proposed methodology is demonstrated using a real-world dataset under fuzzy conditions.

The aim of this work is to tackle the problem of sampling from multi-modal distributions when Hybrid Monte Carlo (HMC) algorithm is employed for performing Bayesian inference in dynamical systems. Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo (MCMC) algorithm but it still suffers from the “multiple peaks” problem. Due to non-trivial structure in the space of (a class of) dynamical systems, posterior distribution of its model parameters could exhibit complicated structures such as multiple ridges. We examined a MCMC algorithm combining HMC with so-called Parallel Tempering (PT) - a well-known strategy for tackling the problem highlighted above. The new algorithm is referred to as PT-HMC. Our numerical experiment demonstrated that when compared to the ground truth, the posterior distributions derived from PT-HMC samples is more accurate than those from HMC.