Bayesian Inference, Model Selection and Likelihood Estimation using Fast Rejection Sampling: The Conway-Maxwell-Poisson Distribution

Regression models with intractable normalizing constants are valuable tools for analyzing complex data structures, yet parameter inference for such models remains highly challenging—particularly when observations are discrete. In statistical inference, discrete state spaces introduce significant computational difficulties, as the normalizing constant often requires summation over extremely large or even infinite sets, which is typically infeasible in practice. These challenges are further compounded when observations are independent but not identically distributed. This paper addresses these issues by developing a novel generalized Bayesian inference approach tailored for regression models with intractable likelihoods. The key idea is to employ a specific form of generalized Fisher divergence to update beliefs about the model parameters, thereby circumventing the need to compute the normalizing constant. The resulting generalized posterior distribution can be sampled using standard computational tools, such as Markov Chain Monte Carlo (MCMC), effectively avoiding the intractability of the normalizing constant.

A Markov Chain Monte Carlo technique for parameter estimation and inference in pesticide fate and transport modeling

Identification of contaminant sources in rivers is crucial for river protection and emergency response. This study presents an innovative approach for identifying river pollution sources by using Bayesian inference and cellular automata (CA) modeling. A general Bayesian framework is proposed that combines the CA model with observed data to identify unknown sources of river pollution. To reduce the computational burden of the Bayesian inference, a CA contaminant transport model is developed to efficiently simulate pollutant concentration values in the river. These simulated concentration values are then used to calculate the likelihood function of available measurements. The Markov chain Monte Carlo (MCMC) method is used to produce the posterior distribution of contaminant source parameters, which is a sampling-based method that enables the estimation of complex posterior distributions. The suggested methodology is applied to a real case study of the Fen River in Yuncheng City, Shanxi Province, Northern China, and it estimates the release time, release mass, and source location with relative errors below 19%. The research indicates that the proposed methodology is an effective and flexible way to identify the location and concentrations of river contaminant sources.

Accurately modeling the growth process of plants in interaction with their environment is important for predicting their biophysical characteristics, referred to as phenotype prediction. Most models are described by discrete dynamic systems in general state-space representation with important domain-specific characteristics: First, plant model parameters have usually clear functional meanings and may be of genetic origins, thus necessitating a precise estimation. Second, critical growth variables, specifically biomass production and dynamic allocation to organs, are hidden variables not accessible to measure. Finally, the difficulty to assess the local plant environment may imply the introduction of process noises in models. Therefore, a precise understanding of the system’s behavior requires the joint estimation of functional parameters, hidden states, and noise parameters. In this paper we describe how a full Bayesian method of estimation can accurately estimate all these key model variables using Markov chain Monte Carlo (MCMC) techniques. In the presence of both process and observation noises, it requires to use adequate particle MCMC (PMCMC) algorithms to efficiently sample the hidden states which, consequently, allows for a precise estimation of all noise parameters involved. Thanks to the Bayesian framework, appropriate choices of prior distributions for the noise parameters have enabled analytical posterior distributions and only simple updates are required. Furthermore, this estimation strategy can be easily generalized and adapted to different types of plant growth models, such as organ-scale or compartmental, provided that they are formulated as hidden Markov models. Our estimation method improves on those classically used in plant growth modeling in several aspects: First, by building upon a general probabilistic framework the estimation results allow proper statistical analyses. It is useful in prediction, no only for uncertainty and risk analysis (e.g., for crop yield prediction) but also to analyze the results of experimental trials, for example, to compare genotypes in breeding. Moreover, the care taken in the estimation of hidden variables opens new perspectives in the understanding of inner growth processes, notably the balance and interaction between biomass production and allocation (referred to as source-sink dynamics). Applications of this estimation procedure are demonstrated on the GreenLab model for Arabidopsis thaliana and the Log-Normal Allocation and Senescence (LNAS) model for sugar beet, on both synthetic and real data.

Sampling technique is one of the asymptotically unbiased estimation approaches for inference in Bayesian probabilistic models. Markov chain Monte Carlo (MCMC) is a kind of sampling methods, which is widely used in the inference of complex probabilistic models. However, current MCMC methods can incur high autocorrelation of samples, which means that the samples generated by MCMC samplers are far from independent. In this paper, we propose the neural networks Langevin Monte Carlo (NNLMC) which makes full use of the flexibility of neural networks and the high efficiency of the Langevin dynamics sampling to construct a new MCMC sampling method. We propose the new update function to generate samples and employ appropriate loss functions to improve the performance of NNLMC during the process of sampling. We evaluate our method on a large diversity of challenging distributions and real datasets. Our results show that NNLMC is able to sample from the target distribution with low autocorrelation and rapid convergence, and outperforms the state-of-the-art MCMC samplers.

Chapter 56 Computationally intensive methods for integration in econometrics

A tutorial introduction to Bayesian inference for stochastic epidemic models using Markov chain Monte Carlo methods

Variational Bayes approximation of factor stochastic volatility models

Appropriate models in biostatistics are often quite complicated. Such models are typically most easily fit using Bayesian methods, which can often be implemented using simulation techniques. Markov chain Monte Carlo (MCMC) methods are an important set of tools for such simulations. We give an overview and references of this rapidly emerging technology along with a relatively simple example. MCMC techniques can be viewed as extensions of iterative maximization techniques, but with random jumps rather than maximizations at each step. Special care is needed when implementing iterative maximization procedures rather than closed-form methods, and even more care is needed with iterative simulation procedures: it is substantially more difficult to monitor convergence to a distribution than to a point. The most reliable implementations of MCMC build upon results from simpler models fit using combinations of maximization algorithms and noniterative simulations, so that the user has a rough idea of the location and scale of the posterior distribution of the quantities of interest under the more complicated model. These concerns with implementation, however, should not deter the biostatistician from using MCMC methods, but rather help to ensure wise use of these powerful techniques.

The Ideal Observer (IO) performance has been advocated when optimizing medical imaging systems for signal detection tasks. However, analytical computation of the IO test statistic is generally intractable. To approximate the IO test statistic, sampling-based methods that employ Markov-Chain Monte Carlo (MCMC) techniques have been developed. However, current applications of MCMC techniques have been limited to several object models such as a lumpy object model and a binary texture model, and it remains unclear how MCMC methods can be implemented with other more sophisticated object models. Deep learning methods that employ generative adversarial networks (GANs) hold great promise to learn stochastic object models (SOMs) from image data. In this study, we described a method to approximate the IO by applying MCMC techniques to SOMs learned by use of GANs. The proposed method can be employed with arbitrary object models that can be learned by use of GANs, thereby the domain of applicability of MCMC techniques for approximating the IO performance is extended. In this study, both signal-known-exactly (SKE) and signal-known-statistically (SKS) binary signal detection tasks are considered. The IO performance computed by the proposed method is compared to that computed by the conventional MCMC method. The advantages of the proposed method are discussed.

Clustering is a fundamental problem in statistics and machine learning, whose solution is commonly computed by the Expectation-Maximization (EM) method, which finds a locally optimal solution for an objective function called log-likelihood. Since the surface of the log-likelihood function is non convex, a stochastic search with Markov Chain Monte Carlo (MCMC) methods can help escaping locally optimal solutions. In this article, we tackle two fundamental conflicting goals: Finding higher quality solutions and achieving faster convergence. With that motivation in mind, we introduce an efficient algorithm that combines elements of the EM and MCMC methods to find clustering solutions that are qualitatively better than those found by the standard EM method. Moreover, our hybrid algorithm allows tuning model parameters and understanding the uncertainty in their estimation. The main issue with MCMC methods is that they generally require a very large number of iterations to explore the posterior of each model parameter. Convergence is accelerated by several algorithmic improvements which include sufficient statistics, simplified model parameter priors, fixing covariance matrices and iterative sampling from small blocks of the data set. A brief experimental evaluation shows promising results.

ABSTRACTA regression model with skew-normal errors provides a useful extension for ordinary normal regression models when the dataset under consideration involves asymmetric outcomes. In this article, we explore the use of Markov Chain Monte Carlo (MCMC) methods to develop a Bayesian analysis for joint location and scale nonlinear models with skew-normal errors, which relax the normality assumption and include the normal one as a special case. The main advantage of these class of distributions is that they have a nice hierarchical representation that allows the implementation of MCMC methods to simulate samples from the joint posterior distribution. Finally, simulation studies and a real example are used to illustrate the proposed methodology.

This article estimates technical and allocative inefficiencies and increase in costs therefrom of individual firms using a translog cost system consisting of the cost function and the cost share equations. We call it a nonlinear random-effects system because technical and allocative inefficiencies are random (hence the term random effects) and are separated from the random noise terms appearing in each equation of the system, and because the inefficiency terms appear in the system in a highly nonlinear fashion, which helps in separating them from random errors. We use Bayesian inference procedures based on Markov chain Monte Carlo (MCMC) techniques to estimate the proposed system. Inferences on firm-specific technical inefficiency and both input-specific and firm-specific allocative inefficiencies are developed using MCMC techniques. We apply the new methods to a sample of 500 U.S. commercial banks. We focus on input allocation problem based on the assumption that banks minimize cost. Empirical results show that cost for the top (bottom) 10% of banks is increased by at least 3% (11%) due to technical inefficiency. In contrast, very few banks are found to be efficient in allocating all the inputs. Costs are increased by 13% on average due to input misallocation. Increase in costs due to both technical and allocative inefficiencies for the top (bottom) 10% of the banks is at least 11% (29%). When translated into dollar figures, this result indicates that elimination of technical and allocative inefficiencies would save the top (bottom) 10% of the banks more than $$.20 ($$3.56) million. We also find that none of the banks in our sample exceeded the efficient scale size, although most of them were operating near their optimum scale (unitary returns to scale).

Using the standard one-zone synchrotron self-Compton model and the Markov chain Monte Carlo (MCMC) technique, we systematically analyse the quasi-simultaneous multiwavelength spectral energy distributions (SEDs) of Mrk 421 and Mrk 501 during states of relatively low activity. With this model in place, a semi-analytical method is developed to examine the uncertainty in jet power estimation caused by degeneracy of the radiative models. The semi-analytical method, in combination with the MCMC technique, allows us to explore the jet properties over a wide range of the variability time-scale. Our results seem to support the following. (i) In both Mrk 421 and Mrk 501, the jets are powered by rapidly rotating black holes. The black hole spin in Mrk 501 could be lower than that in Mrk 421 or possibly they are equal under the assumption of the Blandford–Znajek mechanism. (ii) The energy losses, which could be used to form the large-scale radio structure, are important for reconciling the differences of the kinetic power derived from the observations of the large-scale structure and the SED fitting results. Moreover, the jet energy losses in the propagation are more significant for Mrk 501 than for Mrk 421.

Factors models are commonly used to analyze high-dimensional data in both single-study and multi-study settings. Bayesian inference for such models relies on Markov chain Monte Carlo (MCMC) methods, which scale poorly as the number of studies, observations, or measured variables increase. To address this issue, we propose new variational inference algorithms to approximate the posterior distribution of Bayesian latent factor models using the multiplicative gamma process shrinkage prior. The proposed algorithms provide fast approximate inference at a fraction of the time and memory of MCMC-based implementations while maintaining comparable accuracy in characterizing the data covariance matrix. We conduct extensive simulations to evaluate our proposed algorithms and show their utility in estimating the model for high-dimensional multi-study gene expression data in ovarian cancers. Overall, our proposed approaches enable more efficient and scalable inference for factor models, facilitating their use in high-dimensional settings. An R package VIMSFA implementing our methods is available on GitHub (github.com/blhansen/VI-MSFA). Supplementary materials for this article are available online.