Efficient Bayesian inference of general Gaussian models on large phylogenetic trees

Fast likelihood calculation for multivariate Gaussian phylogenetic models with shifts

<div>We present a framework to solve geophysical inverse problems using the Hamiltonian Monte Carlo (HMC) method, with a focus on Bayesian tomography. Recent work in the geophysical community has shown the potential for gradient-based Monte Carlo sampling for a wide range of inverse problems across several fields.</div><div> </div><div>Many high-dimensional (non-linear) problems in geophysics have readily accessible gradient information which is unused in classical probabilistic inversions. Using HMC is a way to help improve traditional Monte Carlo sampling while increasing the scalability of inference problems, allowing access to uncertainty quantification for problems with many free parameters (>10'000). The result of HMC sampling is a collection of models representing the posterior probability density function, from which not only "best" models can be inferred, but also uncertainties and potentially different plausible scenarios, all compatible with the observed data. However, the amount of tuning parameters required by HMC, as well as the complexity of existing statistical modeling software, has limited the geophysical community in widely adopting a specific tool for performing efficient large-scale Bayesian inference.</div><div> </div><div>This work attempts to make a step towards filling that gap by providing an HMC sampler tailored for geophysical inverse problems (by e.g. supplying relevant priors and visualizations) combined with a set of different forward models, ranging from elastic and acoustic wave propagation to magnetic anomaly modeling, traveltimes, etc.. The framework is coded in the didactic but performant languages Julia and Python, with the possibility for the user to combine their own forward models, which are linked to the sampler routines by proper interfaces. In this way, we hope to illustrate the usefulness and potential of HMC in Bayesian inference. Tutorials featuring an array of physical experiments are written with the aim of both showcasing Bayesian inference and successful HMC usage. It additionally includes examples on how to speed up HMC e.g. with automated tuning techniques and GPU computations.</div>

Efficient Bayesian inference with latent Hamiltonian neural networks in No-U-Turn Sampling

Bayesian attenuation tomography (Tarantola, 1987) is being used to refine existing Lg attenuation models in eastern Asia. The advantages of a Bayesian approach to tomography are that large-scale and high-resolution tomographic models available from other well-accepted studies can be used as prior background models. The resulting refined tomographic model will blend into these prior background models. Moreover, the uncertainties are well established in a Bayesian framework. We assume a general linear Gaussian (least squares) model, where the covariance matrix is partitioned into data and prior model components. Uncertain data are naturally down weighted and model components with small errors will be subject to little change. Amplitude tomography provides only an approximation to the propagation effects a seismic wave may experience. For example, phase blockage can occur over relatively short distances in the absence of any anelastic effects. Station-centric kriged amplitude correction surfaces on top of the tomographic models may assist in identifying blocked paths. Because a signal-to-noise criterion is used to select amplitudes for the inversion, the data are left-censored and the resulting Q 0 models will be biased high. We examine the utility of a maximum-likelihood data augmentation method to the left-censored data problem (Schafer, 1997). In this case, we can only measure an upper bound to measured amplitudes on the basis of prephase noise. Data augmentation is used to impute the missing data values with their conditional expectation based on the relationship of amplitudes with other completely observed variables. We have initially chosen a bivariate Gaussian model for filling in missing amplitudes based on the relationship between amplitude and prephase noise. We apply the technique to eastern Asia for Lg signals at 1 Hz using data from 1651 earthquakes recorded at 12 stations. Tomographic patterns correlate well with those expected from geophysical considerations (e.g., high attenuation in Tibet and low attenuation up into the stable regions of Kazakhstan). Many of the large cratonic basins surrounding the Tibet Plateau (e.g., Tarim, Junggar) show reduced attenuation consistent with the hypothesis that they represent regions of stronger lithosphere. Data augmentation tends to increase attenuation in Tibet (particularly western Tibet) and the cratonic basins to the North of but not to the East of Tibet. In addition, the resulting models are much smoother than models that do not account for censoring.

The effect of missing data on phylogenetic methods is a potentially important issue in our attempts to reconstruct the Tree of Life. If missing data are truly problematic, then it may be unwise to include species in an analysis that lack data for some characters (incomplete taxa) or to include characters that lack data for some species. Given the difficulty of obtaining data from all characters for all taxa (e.g., fossils), missing data might seriously impede efforts to reconstruct a comprehensive phylogeny that includes all species. Fortunately, recent simulations and empirical analyses suggest that missing data cells are not themselves problematic, and that in- complete taxa can be accurately placed as long as the overall number of characters in the analysis is large. How- ever, these studies have so far only been conducted on parsimony, likelihood, and neighbor-joining methods. Although Bayesian phylogenetic methods have become widely used in recent years, the effects of missing data on Bayesian analysis have not been adequately studied. Here, we conduct simulations to test whether Bayesian analyses can accurately place incomplete taxa despite extensive missing data. In agreement with previous studies of other methods, we find that Bayesian analyses can accurately reconstruct the position of highly incomplete taxa (i.e., 95% missing data), as long as the overall number of characters in the analysis is large. These results suggest that highly incomplete taxa can be safely included in many Bayesian phylogenetic analyses. The impact of missing data is a potentially im- portant issue in phylogenetic analyses, particularly if the goal is to reconstruct a comprehensive Tree of Life that includes both fossil and living taxa. Missing data are often encountered when combining data from two or more different genes, when some of the taxa have sequence data available for one gene but not the other. If the taxa lacking data for a gene are included in the combined analysis, then the characters associated with this gene are typically coded as missing or unknown (often denoted with a ?). Similarly, missing data are often encountered in analyses that include fossil taxa, when certain taxa must be scored as unknown for certain characters because the relevant features have not been adequately preserved. Concerns about missing data may often deter- mine what characters and taxa will be included in an analysis (Wiens, 2006), even if this is not always stated explicitly by researchers. For example, if missing data are considered to be problematic, then one should only include species that have complete data for all characters or else only include characters that have complete data for all species. Thus, one may have to reduce the number of taxa or characters in an

In this paper, we consider variable selection for ultra-high dimensional quantile regression model with missing data and measurement errors in covariates. Specifically, we correct the bias in the loss function caused by measurement error by applying the orthogonal quantile regression approach and remove the bias caused by missing data using the inverse probability weighting. A nonconvex Atan penalized estimation method is proposed for simultaneous variable selection and estimation. With the proper choice of the regularization parameter and under some relaxed conditions, we show that the proposed estimate enjoys the oracle properties. The choice of smoothing parameters is also discussed. The performance of the proposed variable selection procedure is assessed by Monte Carlo simulation studies. We further demonstrate the proposed procedure with a breast cancer data set.

This study addresses the challenge of estimating parameters for two logistic populations that share a common scale parameter but have different location parameters in the presence of fuzzy data. To handle these complexities, both Maximum Likelihood Estimation (MLE) and Bayesian methods are employed. Asymptotic confidence intervals are constructed using ML estimates. For Bayesian estimation, a conjugate prior is utilized, and Bayes estimators are approximated using Lindley’s method due to the lack of closed-form solutions. Furthermore, Approximate Bayesian Computation (ABC) and Markov Chain Monte Carlo (MCMC) techniques, including Hamiltonian Monte Carlo (HMC) and the Metropolis–Hastings (MH) algorithm, are utilized to sample from the posterior distributions and construct Highest Posterior Density (HPD) intervals. A detailed comparative analysis of MLE, Lindley’s approximation, ABC, HMC, and MH is conducted to assess their performance. The effectiveness of the proposed methodology is demonstrated using a real-world dataset under fuzzy conditions.

This chapter focuses on measurement error in covariates in regression analyses in which the aim is to estimate the association between one or more covariates and an outcome, adjusting for confounding. Error in covariate measurements, if ignored, results in biased estimates of parameters representing the associations of interest. Studies with variables measured with error can be considered as studies in which the true variable is missing, for either some or all study participants. We make the link between measurement error and missing data and describe methods for correcting for bias due to covariate measurement error with reference to this link, including regression calibration (conditional mean imputation), maximum likelihood and Bayesian methods, and multiple imputation. The methods are illustrated using data from the Third National Health and Nutrition Examination Survey (NHANES III) to investigate the association between the error-prone covariate systolic blood pressure and the hazard of death due to cardiovascular disease, adjusted for several other variables including those subject to missing data. We use multiple imputation and Bayesian approaches that can address both measurement error and missing data simultaneously. Example data and R code are provided in online materials.

Treatment evaluation in advanced cancer mainly relies on overall survival and tumor size dynamics. Both markers and their association can be simultaneously analyzed by using joint models, and these approaches are supported by many softwares or packages. However, these approaches are essentially limited to linear models for the longitudinal part, which limit their biological interpretation. More biological models of tumor dynamics can be obtained by using nonlinear models, but they are limited by the fact that parameter identifiability require rich dataset. In that context Bayesian approaches are particularly suited to incorporate the biological knowledge and increase the information available, but they are limited by the high computing cost of Monte-Carlo by Markov Chains algorithms. Here, we aimed to assess the performances of the Hamiltonian Monte-Carlo (HMC) algorithm implemented in Stan for inference in a nonlinear joint model. The method was validated on simulated data where HMC provided proper posterior distributions and credibility intervals in a reasonable computational time. Then the association between tumor size dynamics and survival was assessed in patients with advanced or metastatic bladder cancer treated with atezolizumab, an immunotherapy agent. HMC confirmed limited sensitivity to prior distributions. A cross-validation approach was developed and identified the current slope of tumor size dynamics as the most relevant driver of survival. In summary, HMC is an efficient approach to perform nonlinear joint models in a Bayesian framework, and opens the way for the use of nonlinear models to characterize both the rapid dynamics and the intersubject variability observed during cancer immunotherapy treatment.

We apply the hybrid Monte Carlo (HMC) algorithm to the financial time sires analysis of the stochastic volatility (SV) model for the first time. The HMC algorithm is used for the Markov chain Monte Carlo (MCMC) update of volatility variables of the SV model in the Bayesian inference. We compute parameters of the SV model from the artificial financial data and compare the results from the HMC algorithm with those from the Metropolis algorithm. We find that the HMC decorrelates the volatility variables faster than the Metropolis algorithm. We also make an empirical analysis based on the Yen/Dollar exchange rates.

Although forecasts of future events are simply uncertain, predicting is one of the most important aspects of future planning. Accurate rice price predictions tend to be helpful for wholesalers, producers, and farmers to develop plans and strategies to reduce the risks that can be faced. Structural time series models are the most plausible alternative for long-term forecasting. This paper proposes an alternate method for modeling average rice prices using structural time series along with Bayesian parameter inference via Hamiltonian Monte Carlo (HMC). The model has been built using the monthly average wholesale rice price from January 2010 to December 2019. For working out both structural time series and HMC, the TensorFlow Probability Library was used. Linear trend, seasonal, and autoregressive components were combined as an additive model to the structural time model. The proposed Hamiltonian parameter produces an optimal acceptance rate. Their trace plot was used to diagnose the convergence of their chain. One of the predictive accuracy of models was assessed using the mean absolute percent error (MAPE). Through both single and multiple chain iterations, the prediction accuracy of a year-ahead is highly accurate, with MAPE less than 2%. Long-term iteration draws during Hamiltonian Monte Carlo should be considered when attempting to achieve more convergence.

In lifestyle intervention trials, where the goal is to change a participant's weight or modify their eating behavior, self-reported diet is a longitudinal outcome variable that is subject to measurement error. We propose a statistical framework for correcting for measurement error in longitudinal self-reported dietary data by combining intervention data with auxiliary data from an external biomarker validation study where both self-reported and recovery biomarkers of dietary intake are available. In this setting, dietary intake measured without error in the intervention trial is missing data and multiple imputation is used to fill in the missing measurements. Since most validation studies are cross-sectional, they do not contain information on whether the nature of the measurement error changes over time or differs between treatment and control groups. We use sensitivity analyses to address the influence of these unverifiable assumptions involving the measurement error process and how they affect inferences regarding the effect of treatment. We apply our methods to self-reported sodium intake from the PREMIER study, a multi-component lifestyle intervention trial.

This paper summarizes the recent developments of hamiltorch,1 which is a PyTorch-based library for Hamiltonian Monte Carlo (HMC). HMC is a gradient-based Markov chain Monte Carlo technique that has favorable scaling properties to high-dimensional parameter spaces, leading to it often becoming the sampling method of choice when derivatives are available. HMC is therefore especially suited to performing Bayesian inference over neural networks (NN). As a result hamiltorch has been specifically developed to sample from PyTorch NN models. The development of hamiltorch has also led to new sampling schemes that are part of the library. In this paper I will highlight such novelties as well as provide a brief introduction to the features of hamiltorch and highlight its use-cases in recently referenced work.

This article reviews the Bayesian inference with the Monte Carlo Markov Chain (MCMC) and the Hamiltonian Monte Carlo (HMC) samplers as a competitor of the classical likelihood statistical inference for pharmacometricians. The MCMC and the HMC samplers have greatly contributed to realization of the Bayesian methods with minimal requirement of mathematical theory. They do not require any closed form of the posterior density nor linear approximation of complex nonlinear models in high dimension even with non-conjugate priors. The HMC even weakens the dependency of the chain and improves computational efficiency. Pharmacometrics is one of great beneficiaries since they use complex multivariate multilevel nonlinear mixed effects models based on the restricted maximum likelihood estimation. Comprehension of the Bayesian approach will help pharmacometricians to access the data analysis more conveniently.

Importance sampling (IS) is a powerful Monte Carlo (MC) methodology for approximating integrals, for instance in the context of Bayesian inference. In IS, the samples are simulated from the so-called proposal distribution, and the choice of this proposal is key for achieving a high performance. In adaptive IS (AIS) methods, a set of proposals is iteratively improved. AIS is a relevant and timely methodology although many limitations remain yet to be overcome, e.g., the curse of dimensionality in high-dimensional and multi-modal problems. Moreover, the Hamiltonian Monte Carlo (HMC) algorithm has become increasingly popular in machine learning and statistics. HMC has several appealing features such as its exploratory behavior, especially in high-dimensional targets, when other methods suffer. In this letter, we introduce the novel Hamiltonian adaptive importance sampling (HAIS) method. HAIS implements a two-step adaptive process with parallel HMC chains that cooperate at each iteration. The proposed HAIS efficiently adapts a population of proposals, extracting the advantages of HMC. HAIS can be understood as a particular instance of the generic layered AIS family with an additional resampling step. HAIS achieves a significant performance improvement in high-dimensional problems w.r.t. state-of-the-art algorithms. We discuss the statistical properties of HAIS and show its high performance in two challenging examples.