Opaque prior distributions in Bayesian latent variable models

A strictly Bayesian model consists of a set of possible data distributions and a prior distribution over that set. If there are other models available, how well they predicted the data may be compared using Bayes factors. If not, a model may be checked using a Bayesian p-value such as a prior predictive p-value or a posterior predictive p-value. However, recent criticisms of ordinary p-values apply with equal force against Bayesian p-values. Many of those criticisms are overcome by e-values, martingales interpreted as the amount of evidence discrediting a null hypothesis, measured as a payoff for betting against it. This article proposes the use of e-values to check Bayesian models by testing their prior predictive distributions as null hypotheses. Two generally applicable methods for checking strictly Bayesian models are provided. The first method calibrates Bayesian p-values by transforming them into Bayesian e-values. The second method uses Bayes factors or their approximations as Bayesian e-values. A robust Bayesian model, a set of strictly Bayesian models, may be checked using various functions that use the e-values of those strictly Bayesian models. Other functions measure how much the data support a Bayesian model. Relations to possibility theory are discussed.

A natural method for approximating out-of-sample predictive evaluation is leave-one-out cross-validation (LOOCV) --- we alternately hold out each case from a full data set and then train a Bayesian model using Markov chain Monte Carlo (MCMC) without the held-out; at last we evaluate the posterior predictive distribution of all cases with their actual observations. However, actual LOOCV is time-consuming. This paper introduces two methods, namely iIS and iWAIC, for approximating LOOCV with only Markov chain samples simulated from a posterior based on a \textit{full} data set. iIS and iWAIC aim at improving the approximations given by importance sampling (IS) and WAIC in Bayesian models with possibly correlated latent variables. In iIS and iWAIC, we first integrate the predictive density over the distribution of the latent variables associated with the held-out without reference to its observation, then apply IS and WAIC approximations to the integrated predictive density. We compare iIS and iWAIC with other approximation methods in three real data examples that respectively use mixture models, models with correlated spatial effects, and a random effect logistic model. Our empirical results show that iIS and iWAIC give substantially better approximates than non-integrated IS and WAIC and other methods.

Objective The Bayesian model plays an important role in diagnostic test evaluation in the absence of the gold standard, which used the external prior distribution of a parameter combined with sample data to yield the posterior distribution of the test characteristics. However, the correlation between diagnostic tests has always been a problem that cannot be ignored in the Bayesian model evaluation. This study will discuss how different Bayesian model, correlation scenarios, and prior distribution affect the outcome. MethodsThe data analyzed in this study was gathered during studies of patients presenting to the Nanjing Chest Hospital with suspected tuberculosis. The diagnostic character of T-SPOT.Tb and KD38 tuberculosis antibody test were evaluated in different Bayesian model, and discharge diagnosis as a gold standard was used to verify the model results in the end. Result The comparison of four models under the conditional independence situation found that Bayesian probabilistic constraint model was consistent with the Conditional Covariance Bayesian model. The results were mainly affected by prior information. The sensitivity and specificity of the two tests in Conditional Covariance Bayesian model in prior constraint situation were considerably higher than the Bayesian probabilistic constraint model in prior constraint situation. The results of the four models under the conditional dependence situation were similar to the conditional independence situation; pD was also negative with no prior constraint situation in both model Bayesian probabilistic constraint model and Conditional Covariance Bayesian model. The Deviance Information Criterion of Bayesian probabilistic constraint model was close to model Conditional Covariance Bayesian model, but pD of Conditional Covariance Bayesian model in Prior constraint situation (pD=2.40) was higher than the Bayesian probabilistic constraint model in Prior constraint situation (pD=1.66). Conclusion The result of Conditional Covariance Bayesian model in prior constraint with conditional independence situation was closest to the result of gold standard evaluation in our data. Both of the two Bayesian methods are the feasible way for the evaluation of diagnostic test in the absence of the gold standard diagnostic. Prior source, priority number, and conditional dependencies should be considered in the method selection, the accuracy of posterior estimation mainly depending on the prior distribution.

Bayesian calibration, comparison and averaging of six forest models, using data from Scots pine stands across Europe

In spite of over half a century of research, little is known about the genetic basis of developmental instability (DI). The estimation of the heritability of DI is seriously hampered by the fact that fluctuating asymmetry-FA, that is, the observable outcome of DI-only poorly reflects DI. This results in an underestimation of the heritability of DI. Current methods transforming heritabilities in FA into those of DI fail to take all sources of variation into account and yield incorrect confidence bands that are usually based on unrealistic assumptions. Therefore, a Bayesian latent variable model is developed and explored. Simulations show that with sample sizes currently applied in empirical studies, extremely wide posterior distributions are obtained and data do not allow to distinguish between high (0.5) and low (0.1) heritabilities of DI at all. Even sample sizes of 5000 result in very wide posteriors in many cases. Furthermore, for smaller samples (250 and 1250), up to 70% of the estimates of the heritability of DI were below the mean expected value because of the high skewness of its distribution. Knowing that in only one study, sample sizes were above 5000, there is a need for larger studies to evaluate the evolutionary potential of DI. Designs with relatively low numbers of sires (1-2% of total number of offspring) appear most efficient. Because such high sample sizes are hard to obtain for many study organisms, more insights are required about how data from different traits can be combined in a single analysis. In addition, new designs and methods, such as QTL analyses and micro-array techniques, should be applied to gain a better understanding of the genetic basis of DI.

Strange bayes indeed: uniform topological priors imply non-uniform clade priors

We discuss the problem of model criticism, with emphasis on developing summary diagnostic measures. We approach model criticism by identifying possible troublesome features of the currently entertained model, embedding the model in an elaborated model, and measuring the value of elaborating. This requires three elements: a model elaboration, a prior distribution, and a utility function. Each triplet generates a different diagnostic measure. We focus primarily on the measure given by a Kullback—Leibler divergence between the marginal prior and posterior distributions on the elaboration parameter. We also develop a linearized version of this diagnostic and use it to show that our procedure is related to other tools commonly used for model diagnostics, such as Bayes factors and the score function. One attraction of this approach is that it allows model criticism to be performed jointly with parameter inference and prediction. Also, this diagnostic approach aims at maintaining an exploratory nature t...

From an algorithmic complexity point of view, machine learning methods scale and generalize better using a few key features; using lots is computationally expensive, and overfitting can occur. High dimensional data is often counterintuitive to perceive and process, but unfortunately it is common for observed data to be in a representation of greater dimensionality than it requires. This gives rise to the notion of dimensionality reduction, a sub-field of machine learning that is motivated to find a descriptive low-dimensional representation of data. In this review, it is explored a way to perform dimensionality reduction, provided by a class of Latent Variable Models (LVMs). In particular, the aim is to establish the technical foundations required for understanding the Gaussian Process Latent Variable Model (GP-LVM), a probabilistic nonlinear dimensionality reduction model. The review is organized as follows: after an introduction to the problem of dimensionality reduction and LVMs, Principal Component Analysis (PCA) is recalled and it is reviewed its probabilistic equivalent that contributes to the derivation of GP-LVM. Then, GP-LVM is introduced, and briefly a remarkable extension of the latter, the Bayesian Gaussian Process Latent Variable Model (BGP-LVM) is described. Eventually, and the main advantages of using GP-LVM are summarized.

Among the multi-trait models selected to study several traits and environments jointly, the Bayesian framework has been a preferred tool when constructing a more complex and biologically realistic model. In most cases, non-informative prior distributions are adopted in studies using the Bayesian approach. However, the Bayesian approach presents more accurate estimates when informative prior distributions are used. The present study was developed to evaluate the efficiency and applicability of multi-trait multi-environment (MTME) models within a Bayesian framework utilizing a strategy for eliciting informative prior distribution using previous data on rice. The study involved data pertaining to rice (Oryza sativa L.) genotypes in three environments and five crop seasons (2010/2011 until 2014/2015) for the following traits: grain yield (GY), flowering in days (FLOR) and plant height (PH). Variance components, genetic and non-genetic parameters were estimated using the Bayesian method. In general, the informative prior distribution in Bayesian MTME models provided higher estimates of individual narrow-sense heritability and variance components, as well as minor lengths for the highest probability density interval (HPD), compared to their respective non-informative prior distribution analyses. More informative prior distributions make it possible to detect genetic correlations between traits, which cannot be achieved with non-informative prior distributions. Therefore, this mechanism presented to update knowledge for an elicitation of an informative prior distribution can be efficiently applied in rice breeding programs.

We propose a Bayesian model selection (BMS) boundary detection procedure using non-local prior distributions for a sequence of data with multiple systematic mean changes. By using the non-local priors in the BMS framework, the BMS method can effectively suppress the non-boundary spike points with large instantaneous changes. Further, we speedup the algorithm by reducing the multiple change points to a series of single change point detection problems. We establish the consistency of the estimated number and locations of the change points under various prior distributions. From both theoretical and numerical perspectives, we show that the non-local inverse moment prior leads to the fastest convergence rate in identifying the true change points on the boundaries. Extensive simulation studies are conducted to compare the BMS with existing methods, and our method is illustrated with application to the magnetic resonance imaging guided radiation therapy data.

Equality and inequality constrained multivariate linear models: Objective model selection using constrained posterior priors

The dynamic nature of the ocean environment presents numerous challenges to parameter estimation in ocean acoustics. We present a variational Bayesian method which selects an optimal model for parameter estimation in ocean acoustics using Gaussian process (GP) regression. GP model selection is based on Bayesian model selection, which treats the parameters and kernel selection of the model as hyperparameters and uses Bayesian inference to compute a model’s likelihood with respect to its hyperparameters. However, in Bayesian model selection, the integrals over parameter space are often intractable and require approximation or sampling methods such as Markov chain Monte Carlo. GP renders these integrals tractable by treating hyperparameters as multivariate Gaussian distributions, which greatly speeds up the optimization. Furthermore, model selection using GPs allows for straightforward evaluation of uncertainty in the selected model’s likelihood. In this study, we compare results of acoustic parameter estimation from Bayesian and GP model selection using an ocean acoustic propagation model.

The creep deformation process is constructed by complex interactions of multiple factors, and the measurement of creep deformation requires enormous economic costs and a long experimental time, so there is a small amount of measurement data. In such a situation, multiple models are often proposed to explain the same experimental data. The coexistence of multiple models based on different physical assumptions makes it difficult to understand the creep deformation process. The purpose of this study is to construct a framework to compare and evaluate coexistence models based on measurement data using the Bayesian model selection framework. Basically, in the creep deformation model, basis functions, usually the exponential function eat or power function tb, corresponding to the primary creep stage and the tertiary creep stage are set, and the time series of the whole creep deformation process is regressed as the linear sum of these bases. The parameters a and b of the basis functions are parameters that must also be optimized to regress the time series data. Izuno et al. proposed the Bayesian model selection method for the creep deformation process, and it was confirmed that the model with the linear term corresponding to the secondary creep stage, called the modified theta method, is a good model compared to the model without adding the linear term. To obtain the criterion of Bayesian model selection analytically, which can reduce the computational cost greatly, Izuno et al. treated only the regression coefficients as the probabilistic parameter of the creep model. In this method, parameters a and b of the basis functions are optimized by a grid search. Because the grid search requires significant computational costs if multiple basis functions are set for each creep stage, as in the model proposed by Kimura et al., it is difficult to apply the algorithm to models with multiple bases. In addition, if parameters a and b of the basis functions correlate with the regression coefficients, the method of Izuno et al. would offer a biased evaluation of the models. To improve such an existing method, we proposed a Bayesian model selection method that treats the parameters of the basis functions as probabilistic parameters. The large computational cost to treat the parameters of the basis functions as probabilistic parameters was reduced using the exchange Monte Carlo method, which is an efficient sampling method. By applying our proposed method to the evaluation of the creep deformation model, information was obtained to understand the creep deformation process deeply. For example, it was confirmed that there exists a model in which there is a correlation between the parameter of basis function and the regression coefficients.

When rejecting and accepting hypotheses, it is important to determine the risk of false positive conclusions. Hierarchical Bayesian models are frequently applied in cognitive science, psychology, cognitive neuroscience, and psychiatry to reduce the variability of low-level parameters through partial pooling, and to obtain more robust parameter estimates for complex models using limited data. This may lower the risk of false positive inferences from lower-level estimates. In contrast, higher-level (e.g. group-level) estimates may be more affected by modeling choices (e.g. prior distributions), population-level variances in effects, and inference methods, which may in turn affect false positive rates for group-level effects. Here we used simulation and computational modeling to systematically examine false positive rates and effects of specific modeling choices on inferences in a simple hierarchical Bayesian temporal discounting model. To this end, 600 data sets of a temporal discounting task with two conditions were simulated, drawing individual condition effect parameters from a null distribution with varying standard deviations. For each simulated data set, the group-level mean change in the discount rate was modeled with eight different prior variances of the condition effect. Results were subsequently evaluated using several commonly applied Bayesian inference methods: the Savage-Dickey Bayes factor (BF), the directional Bayes factor (dBF), P(effect > 0), and the highest density interval (HDI) against zero decision rule. Results confirm that inference methods that directly depend on the posterior distribution can be used regardless of prior variances, whereas the Savage-Dickey BF should only be used with sufficiently informative priors. Based on the simulation work, decision thresholds that ensure false positive rates below 5% are provided for each inference method. Results are discussed in the context of computational modeling approaches in cognitive (neuro-)science and psychology.

In statistical practice, researchers commonly focus on patterns in the means of multiple dependent outcomes while treating variances as nuisance parameters. However, in fact, there are often substantive reasons to expect certain patterns in the variances of dependent outcomes as well. For example, in a repeated measures study, one may expect the variance of the outcome to increase over time if the difference between subjects becomes more pronounced over time because the subjects respond differently to a given treatment. Such expectations can be formulated as order constrained hypotheses on the variances of the dependent outcomes. Currently, however, no methods exist for testing such hypotheses in a direct manner. To fill this gap, we develop a Bayes factor for this challenging testing problem. Our Bayes factor is based on the multivariate normal distribution with an unstructured covariance matrix, which is often used to model dependent outcomes. Order constrained hypotheses can then be formulated on the variances on the diagonal of the covariance matrix. To compute Bayes factors between multiple order constrained hypotheses, a prior distribution needs to be specified under every hypothesis to be tested. Here, we use the encompassing prior approach in which priors under order constrained hypotheses are truncations of the prior under the unconstrained hypothesis. The resulting Bayes factor is fully automatic in the sense that no subjective priors need to be specified by the user.