Bayesian Inference for Elliptical Linear Models: Conjugate Analysis and Model Comparison

Bayesian sensitivity analysis in elliptical linear regression models

Finding a new drug which is both safe and efficient is an expensive and time-consuming endeavour. In particular, establishing the ‘dose-effect relationship’ – how beneficial a drug is at different dosages – can be challenging. Predicting this curve requires gathering experimental data by exposing and recording how cells respond to various levels of the drug. However, extreme values are often observed at low and high dosages, potentially introducing errors that are hard to correct in the prediction process. Yet, these extreme observations are sometimes genuine so researchers cannot just ignore them. To improve dose-effect estimation, Zhou, Liu, Fang et al. developed a new general-purpose approach. It uses advanced statistical modelling to account for extremes in lab data. This strategy outperformed other methods when dealing with these observations while also providing higher efficiency in data analysis with more uniform data in experiments. To facilitate implementation, Zhou, Liu, Fang et al. set up a user-friendly tool baptized ‘REAP’; this free online resource allows scientists without advanced statistical experience to harness the new approach and to perform dose-effect analysis more easily and accurately. This could boost research across many different disciplines that examine the effects of chemicals on cells.

Chapter 56 Computationally intensive methods for integration in econometrics

Identification of contaminant sources in rivers is crucial for river protection and emergency response. This study presents an innovative approach for identifying river pollution sources by using Bayesian inference and cellular automata (CA) modeling. A general Bayesian framework is proposed that combines the CA model with observed data to identify unknown sources of river pollution. To reduce the computational burden of the Bayesian inference, a CA contaminant transport model is developed to efficiently simulate pollutant concentration values in the river. These simulated concentration values are then used to calculate the likelihood function of available measurements. The Markov chain Monte Carlo (MCMC) method is used to produce the posterior distribution of contaminant source parameters, which is a sampling-based method that enables the estimation of complex posterior distributions. The suggested methodology is applied to a real case study of the Fen River in Yuncheng City, Shanxi Province, Northern China, and it estimates the release time, release mass, and source location with relative errors below 19%. The research indicates that the proposed methodology is an effective and flexible way to identify the location and concentrations of river contaminant sources.

In this work, a simulation study is conducted to evaluate the performance of Bayesian estimators for the log–linear exponential regression model under different levels of censoring and degrees of collinearity for two covariates. The diffuse normal, independent Student-t and multivariate Student-t distributions are considered as prior distributions and to draw from the posterior distributions, the Metropolis algorithm is implemented. Also, the results are compared with the maximum likelihood estimators in terms of the mean squared error, coverages and length of the credibility and confidence intervals.

This dissertation consists of five chapters with three distinct but related research projects. In Chapter 1, we introduce some necessary definitions related to the research work. In Chapter 2, we develop Bayes factor based testing procedures for a general linear hypothesis of the regression coefficients in the context of the normal linear models. We propose two calibration schemes to deal with asymmetry in information of Bayes factor: (i) by controlling Type I error probability of the Bayes factor and (ii) by balancing Type I and Type II error probabilities of the Bayes factor. We evaluate the finite sample performance of the proposed Bayes factors via simulation studies and a real-data application. Experimental results have shown than the proposed Bayes factors perform well in testing the general linear hypothesis of the regression coefficient. In Chapter 3, we consider Bayesian quantile analysis for testing constrained hypotheses in linear models, in which the quantiles the parameters satisfy a simple order restriction. We develop a Bayesian hierarchical model based on the specification the asymmetric Laplace distribution for the error component. We propose a non-iterative sampling algorithm in the Expectation-Maximization (EM) structure to generate independently and identically distributed posterior samples from their posterior distributions of the parameters. Then we adopt the Savage-Dickey density ratios to conduct the multiple comparison with simply order constraints. Simulation studies were conducted to compare the finite sample performance of the proposed non-iterative sampling algorithm with the Gibbs sampling algorithm. In Chapter 4, we consider objective Bayesian analysis for the concordance correlation coefficient (CCC), which is one of the most commonly used metrics to assess agreement of different methods in many practical applications. We develop an objective Bayesian framework for estimating the CCC based a combined use of the multivariate student's t-distribution with noninformative Independence Jeffreys prior for the unknown parameters. Extensive simulation studies are conducted to compare the performance of the proposed Bayesian estimates with the ones under the subjective priors in the literature. In Chapter 5, we discuss some ongoing projects related to our research work mentioned above and some interesting problems for future work.

Bayesian inference for models with intractable likelihood functions represents a challenging suite of problems in modern statistics. In this work we analyse the Conway-Maxwell-Poisson (COM-Poisson) distribution, a two parameter generalisation of the Poisson distribution. COM-Poisson regression modelling allows the flexibility to model dispersed count data as part of a generalised linear model (GLM) with a COM-Poisson response, where exogenous covariates control the mean and dispersion level of the response. The major difficulty with COM-Poisson regression is that the likelihood function contains multiple intractable normalising constants and is not amenable to standard inference and Markov Chain Monte Carlo (MCMC) techniques. Recent work by Chanialidis et al. (2018) has seen the development of a sampler to draw random variates from the COM-Poisson likelihood using a rejection sampling algorithm. We provide a new rejection sampler for the COM-Poisson distribution which significantly reduces the central processing unit (CPU) time required to perform inference for COM-Poisson regression models. An extension of this work shows that for any intractable likelihood function with an associated rejection sampler it is possible to construct unbiased estimators of the intractable likelihood which proves useful for model selection or for use within pseudo-marginal MCMC algorithms (Andrieu and Roberts, 2009). We demonstrate all of these methods on a real-world dataset of takeover bids.

Regression models with intractable normalizing constants are valuable tools for analyzing complex data structures, yet parameter inference for such models remains highly challenging—particularly when observations are discrete. In statistical inference, discrete state spaces introduce significant computational difficulties, as the normalizing constant often requires summation over extremely large or even infinite sets, which is typically infeasible in practice. These challenges are further compounded when observations are independent but not identically distributed. This paper addresses these issues by developing a novel generalized Bayesian inference approach tailored for regression models with intractable likelihoods. The key idea is to employ a specific form of generalized Fisher divergence to update beliefs about the model parameters, thereby circumventing the need to compute the normalizing constant. The resulting generalized posterior distribution can be sampled using standard computational tools, such as Markov Chain Monte Carlo (MCMC), effectively avoiding the intractability of the normalizing constant.

This article discusses the Bayesian approach for count data using non-homogeneous Poisson processes, considering different prior distributions for the model parameters. A Bayesian approach using Markov Chain Monte Carlo (MCMC) simulation methods for this model was first introduced by [1], taking into account software reliability data and considering non-informative prior distributions for the parameters of the model. With the non-informative prior distributions presented by these authors, computational difficulties may occur when using MCMC methods. This article considers different prior distributions for the parameters of the proposed model, and studies the effect of such prior distributions on the convergence and accuracy of the results. In order to illustrate the proposed methodology, two examples are considered: the first one has simulated data, and the second has a set of data for pollution issues at a region in Mexico City.

ABSTRACTSinh-normal/independent distributions are a class of symmetric heavy-tailed distributions that include the sinh-normal distribution as a special case, which has been used extensively in Birnbaum–Saunders regression models. Here, we explore the use of Markov Chain Monte Carlo methods to develop a Bayesian analysis in nonlinear regression models when Sinh-normal/independent distributions are assumed for the random errors term, and it provides a robust alternative to the sinh-normal nonlinear regression model. Bayesian mechanisms for parameter estimation, residual analysis and influence diagnostics are then developed, which extend the results of Farias and Lemonte [Bayesian inference for the Birnbaum-Saunders nonlinear regression model, Stat. Methods Appl. 20 (2011), pp. 423-438] who used the Sinh-normal/independent distributions with known scale parameter. Some special cases, based on the sinh-Student-t (sinh-St), sinh-slash (sinh-SL) and sinh-contaminated normal (sinh-CN) distributions are discussed in detail. Two real datasets are finally analyzed to illustrate the developed procedures.

ABSTRACTA regression model with skew-normal errors provides a useful extension for ordinary normal regression models when the dataset under consideration involves asymmetric outcomes. In this article, we explore the use of Markov Chain Monte Carlo (MCMC) methods to develop a Bayesian analysis for joint location and scale nonlinear models with skew-normal errors, which relax the normality assumption and include the normal one as a special case. The main advantage of these class of distributions is that they have a nice hierarchical representation that allows the implementation of MCMC methods to simulate samples from the joint posterior distribution. Finally, simulation studies and a real example are used to illustrate the proposed methodology.

Parameter estimation and associated uncertainty quantification is an important problem in dynamical systems characterised by ordinary differential equation (ODE) models that are often nonlinear. Typically, such models have analytically intractable trajectories which result in likelihoods and posterior distributions that are similarly intractable. Bayesian inference for ODE systems via simulation methods require numerical approximations to produce inference with high accuracy at a cost of heavy computational power and slow convergence. At the same time, Artificial Neural Networks (ANN) offer tractability that can be utilized to construct an approximate but tractable likelihood and posterior distribution. In this paper we propose a hybrid approach, where Laplace-based Bayesian inference is combined with an ANN architecture for obtaining approximations to the ODE trajectories as a function of the unknown initial values and system parameters. Suitable choices of customized loss functions are proposed to fine tune the approximated ODE trajectories and the subsequent Laplace approximation procedure. The effectiveness of our proposed methods is demonstrated using an epidemiological system with non-analytical solutions—the Susceptible-Infectious-Removed (SIR) model for infectious diseases—based on simulated and real-life influenza datasets. The novelty and attractiveness of our proposed approach include (i) a new development of Bayesian inference using ANN architectures for ODE based dynamical systems, and (ii) a computationally fast posterior inference by avoiding convergence issues of benchmark Markov Chain Monte Carlo methods. These two features establish the developed approach as an accurate alternative to traditional Bayesian computational methods, with improved computational cost.

This paper reconciles the asymptotic disagreement between Bayesian and frequentist inference in set-identified models by adopting a multiple-prior (robust) Bayesian approach. We propose new tools for Bayesian inference in set-identified models. We show that these tools have a well-defined posterior interpretation in finite samples and are asymptotically valid from the frequentist perspective. The main idea is to construct a prior class that removes the source of the disagreement: the need to specify an unrevisable prior. The corresponding class of posteriors can be summarized by reporting the "posterior lower and upper probabilities" of a given event and/or the "set of posterior means" and the associated "robust credible region". We show that the set of posterior means is a consistent estimator of the true identified set and the robust credible region has the correct frequentist asymptotic coverage for the true identified set if it is convex. Otherwise, the method can be interpreted as providing posterior inference about the convex hull of the identified set. For impulse-response analysis in set-identified Structural Vector Autoregressions, the new tools can be used to overcome or quantify the sensitivity of standard Bayesian inference to the choice of an unrevisable prior.

This paper reconciles the asymptotic disagreement between Bayesian and frequentist inference in set-identified models by adopting a multiple-prior (robust) Bayesian approach. We propose new tools for Bayesian inference in set-identified models and show that they have a well-defined posterior interpretation infinite samples and are asymptotically valid from the frequentist perspective. The main idea is to construct a prior class that removes the source of the disagreement: the need to specify an unrevisable prior for the structural parameter given the reduced-form parameter. The corresponding class of posteriors can be summarized by reporting the 'posterior lower and upper probabilities' of a given event and/or the 'set of posterior means' and the associated 'robust credible region'. We show that the set of posterior means is a consistent estimator of the true identi?ed set and the robust credible region has the correct frequentist asymptotic coverage for the true identified set if it is convex. Otherwise, the method provides posterior inference about the convex hull of the identified set. For impulse-response analysis in set-identified Structural Vector Autoregressions, the new tools can be used to overcome or quantify the sensitivity of standard Bayesian inference to the choice of an unrevisable prior.

The evaluation of the mechanical behaviour of timber beams or glued laminated timber lamellas in-service are generally a difficult task due to the different sources of uncertainty involved (small knowledge about the initial quality of timber, small samples, models uncertainty, human errors). The use of statistical methods that can incorporate part of the uncertainty are probably a suitable way to ensure that the predictions made could provide a reliable prediction of the desired property. In most situations while performing in situ assessment of timber structures, the application of non or semi-destructive testing (NDT or SDT) methods relies on regression linear models showing noticeable different coefficients of determination. Another source of uncertainty happens when making in-situ testing relying on the application of existing regression models to timber members without being sure about the wood species or the origin of the wood species. Can these models be used when it is commonly accepted that knowledge on timber’s origin and species have a major impact on the capability to predict strength and stiffness? To comply with uncertainty several studies have been trying to use statistical methods that can incorporate prior information (e.g. Bayesian methods) or uncertainty (e.g. Markov chain Monte Carlo—MCMC). In the present paper uncertainty associated to the use of linear regression models are discussed using as example the prediction of static modulus of elasticity from dynamic modulus of elasticity. For that purpose, data taken from literature and from studies conducted at LNEC are compared, analysed and discussed having in mind to verify the utility of the application of Bayesian linear regression approach and Monte Carlo Markov Chains (MCMC) estimation.