A Bayesian Approach to Some Best Population Problems

This article deals with the issue of using a suitable pseudo-likelihood, instead of an integrated likelihood, when performing Bayesian inference about a scalar parameter of interest in the presence of nuisance parameters. The proposed approach has the advantages of avoiding the elicitation on the nuisance parameters and the computation of multidimensional integrals. Moreover, it is particularly useful when it is difficult, or even impractical, to write the full likelihood function. We focus on Bayesian inference about a scalar regression coefficient in various regression models. First, in the context of non-normal regression-scale models, we give a theroetical result showing that there is no loss of information about the parameter of interest when using a posterior distribution derived from a pseudo-likelihood instead of the correct posterior distribution. Second, we present non trivial applications with high-dimensional, or even infinite-dimensional, nuisance parameters in the context of nonlinear normal heteroscedastic regression models, and of models for binary outcomes and count data, accounting also for possibile overdispersion. In all these situtations, we show that non Bayesian methods for eliminating nuisance parameters can be usefully incorporated into a one-parameter Bayesian analysis.

Bayesian model updating provides a rigorous framework to account for uncertainty induced by lack of knowledge about engineering systems in their respective mathematical models through updates of the joint probability density function (PDF), the so-called posterior PDF, of the unknown model parameters. The Markov chain Monte Carlo (MCMC) methods are currently the most popular approaches for generating samples from the posterior PDF. However, these methods often found wanting when sampling from difficult distributions (e.g., high-dimensional PDFs, PDFs with flat manifolds, multimodal PDFs, and very peaked PDFs). This paper introduces a new multi-level sampling approach for Bayesian model updating, called Sequential Gauss-Newton algorithm, which is inspired by the Transitional Markov chain Monte Carlo (TMCMC) algorithm. The Sequential Gauss-Newton algorithm improves two aspects of TMCMC to make an efficient and effective MCMC algorithm for drawing samples from difficult posterior PDFs. First, the statistical efficiency of the algorithm is enhanced by use of the systematic resampling scheme. Second, a new MCMC algorithm, called Gauss-Newton MCMC algorithm, is proposed which is essentially an M-H algorithm with a Gaussian proposal PDF tailored to the posterior PDF using the gradient and Hessian information of the negative log posterior. The effectiveness of the proposed algorithm for solving the Bayesian model updating problem is illustrated using three examples with irregularly shaped posterior PDFs.

BackgroundTranslating a known metabolic network into a dynamic model requires reasonable guesses of all enzyme parameters. In Bayesian parameter estimation, model parameters are described by a posterior probability distribution, which scores the potential parameter sets, showing how well each of them agrees with the data and with the prior assumptions made.ResultsWe compute posterior distributions of kinetic parameters within a Bayesian framework, based on integration of kinetic, thermodynamic, metabolic, and proteomic data. The structure of the metabolic system (i.e., stoichiometries and enzyme regulation) needs to be known, and the reactions are modelled by convenience kinetics with thermodynamically independent parameters. The parameter posterior is computed in two separate steps: a first posterior summarises the available data on enzyme kinetic parameters; an improved second posterior is obtained by integrating metabolic fluxes, concentrations, and enzyme concentrations for one or more steady states. The data can be heterogenous, incomplete, and uncertain, and the posterior is approximated by a multivariate log-normal distribution. We apply the method to a model of the threonine synthesis pathway: the integration of metabolic data has little effect on the marginal posterior distributions of individual model parameters. Nevertheless, it leads to strong correlations between the parameters in the joint posterior distribution, which greatly improve the model predictions by the following Monte-Carlo simulations.ConclusionWe present a standardised method to translate metabolic networks into dynamic models. To determine the model parameters, evidence from various experimental data is combined and weighted using Bayesian parameter estimation. The resulting posterior parameter distribution describes a statistical ensemble of parameter sets; the parameter variances and correlations can account for missing knowledge, measurement uncertainties, or biological variability. The posterior distribution can be used to sample model instances and to obtain probabilistic statements about the model's dynamic behaviour.

Statistical reliability practice from sampling theory to stochastic filtering

ABSTRACTIn this paper, we study an opportunistic scheduling problem in an OFDMA system, in which sub‐carriers of the system are allocated to each user in each time slot considering the time‐varying channel condition and QoS requirement of each user. We consider two different classes of services that are represented with different types of utility functions. The utility function for a user in one class is defined as a function of its average data rate, which can be applicable to best‐effort services and the utility function for a user in the other class is defined as a function of its instantaneous data rate, which can be applicable to rate‐sensitive services. Those two types of utility functions have been extensively considered in opportunistic scheduling in wireless networks. However, in most of the previous work, they are considered separately in different problems. In this paper, we formulate a stochastic optimization problem that can treat those two types of utility functions in a single problem, which enables us to implement an opportunistic scheduling algorithm that can consider those two classes of services in a single system in a unified way. Through simulations, we first show that our algorithm provides a good approximation to the optimal solution. In addition, we also verify the appropriateness of our utility models. Copyright © 2010 John Wiley & Sons, Ltd.

We present a Monte Carlo approach to estimation of the recombination fraction theta and the profile likelihood for a dichotomous trait and a single marker gene with 2 alleles. The method is an application of a technique known as 'Gibbs sampling', in which random samples of each of the unknowns (here genotypes, theta and nuisance parameters, including the allele frequencies and the penetrances) are drawn from their posterior distributions, given the data and the current values of all the other unknowns. Upon convergence, the resulting samples derive from the marginal distribution of all the unknowns, given only the data, so that the uncertainty in the specification of the nuisance parameters is reflected in the variance of the posterior distribution of theta. Prior knowledge about the distribution of theta and the nuisance parameters can be incorporated using a Bayesian approach, but adoption of a flat prior for theta and point priors for the nuisance parameters would correspond to the standard likelihood approach. The method is easy to program, runs quickly on a microcomputer, and could be generalized to multiple alleles, multipoint linkage, continuous phenotypes and more complex models of disease etiology. The basic approach is illustrated by application to data on cholesterol levels and an a low-density lipoprotein receptor gene in a single large pedigree.

Approximated sensitivity analysis in posterior predictive distribution

Binary-discrimination tasks useful to arms-control-treaty verification were performed by applying mathematical observer models used by the medical-imaging community. It is a difficult task, as the monitor needs to verify a measured object is a warhead while the host wants to prevent dissemination of sensitive information on their weapons. Inspection objects were classified by their projection data, without reconstructing an image, which often contains sensitive information. Furthermore, the models process data event-by-event, with only a scalar test statistic being updated before the observed data is purged from memory, preventing the aggregation of information which would be sensitive. Template matching models were developed from a set of calibration data on these objects, with the ultimate goal being to develop an observer model that stores only non-sensitive information sufficient for confirmation. These models were also analyzed in the presence of nuisance parameters — unknowns in the objects or imaging system that affect the projection data but aren't of interest to the discrimination task. Examples of these include object location and orientation. The Hotelling observer and channelized Hotelling observer were modeled and their benefits to information security analyzed. The Hotelling observer-the ideal linear observer when the statistics of the data are Gaussian-stores only a set of weights which are the product of the inverse average covariance matrix and difference in mean data between the two objects in the discrimination task. When testing a source, an inner product of the Hotelling weights and binned testing data is taken, resulting in a scalar that is thresholded to make a decision. If nuisance parameters are present, the mean and covariance matrix are found by averaging not only over the always present Poisson noise, but the nuisance parameter distributions as well. Hence, even if detector data is gathered from multiple realizations of an object, the Hotelling weights will be a single smeared out data set the size of the measured data. The Hotelling weights also sifts out all information other than the differences between the two objects. Hence, if the monitor was to gain access to the weights and apply the inverse to their test statistic, they could only back out a scaled version of the template, not the image. The channelized Hotelling observer-a common tool used in medical image quality assessment-was also investigated. This method drastically reduces the size of the data by applying a channeiizing matrix to the binned testing data set. An optimal set of weights for the channelized data can then be found, and the inner product between these weights and the channelized vector results in the scalar test statistic. Optimal performance is retained by optimizing the matrix to maximize the SNR2 between the test-statistic distributions for the two sources in the task. The channelized Hotelling observer gives the monitor access to multiple non-sensitive test statistics. In practice, the channelizing matrix could be implemented in hardware or software behind an information barrier. The addition of penalty terms to the channelized Hotelling observer offers further potential for this method. Individual channel performance could be penalized, leading to a large number of non-sensitive channels that the monitor can use to verify the channelization routine is working as described without gaining access to sensitive information. Noise could be added to the resulting channels, causing additional reduction of the total stored information. Finally, if the host can define what information in its object geometries is sensitive in advance, they could optimize the differences between the two objects in the task while penalizing out the sensitive information, creating a non-sensitive channeiizing matrix that could be shared with the monitor. To test these models, Monte Carlo simulations were performed with the GEANT4 toolkit. Photons were tracked from plutonium inspection objects developed by Idaho National Laboratory. We simulated the Fast-Neutron Imaging system designed by Oak Ridge National Laboratory and Sandia National Laboratories, which consists of 40×40 1cm2 liquid scintillator pixels with a plastic coded aperture. Observer models were evaluated using the area under the ROC curve. This work is supported by the Office of Defense Nuclear Nonproliferation Research and Development, Nuclear Weapon and Material Security Team. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-AC04-94AL85000. (SAND2015-10391A).

A crucial decision in designing a spatial sample for soil survey is the number of sampling locations required to answer, with sufficient accuracy and precision, the questions posed by decision makers at different levels of geographic aggregation. In the Indian Soil Health Card (SHC) scheme, many thousands of locations are sampled per district. In this paper the SHC data are used to estimate the mean of a soil property within a defined study area, e.g., a district, or the areal fraction of the study area where some condition is satisfied, e.g., exceedence of a critical level. The central question is whether this large sample size is needed for this aim. The sample size required for a given maximum length of a confidence interval can be computed with formulas from classical sampling theory, using a prior estimate of the variance of the property of interest within the study area. Similarly, for the areal fraction a prior estimate of this fraction is required. In practice we are uncertain about these prior estimates, and our uncertainty is not accounted for in classical sample size determination (SSD). This deficiency can be overcome with a Bayesian approach, in which the prior estimate of the variance or areal fraction is replaced by a prior distribution. Once new data from the sample are available, this prior distribution is updated to a posterior distribution using Bayes’ rule. The apparent problem with a Bayesian approach prior to a sampling campaign is that the data are not yet available. This dilemma can be solved by computing, for a given sample size, the predictive distribution of the data, given a prior distribution on the population and design parameter. Thus we do not have a single vector with data values, but a finite or infinite set of possible data vectors. As a consequence, we have as many posterior distribution functions as we have data vectors. This leads to a probability distribution of lengths or coverages of Bayesian credible intervals, from which various criteria for SSD can be derived. Besides the fully Bayesian approach, a mixed Bayesian-likelihood approach for SSD is available. This is of interest when, after the data have been collected, we prefer to estimate the mean from these data only, using the frequentist approach, ignoring the prior distribution. The fully Bayesian and mixed Bayesian-likelihood approach are illustrated for estimating the mean of log-transformed Zn and the areal fraction with Zn-deficiency, defined as Zn concentration <0.9 mg kg −1, in the thirteen districts of Andhra Pradesh state. The SHC data from 2015–2017 are used to derive prior distributions. For all districts the Bayesian and mixed Bayesian-likelihood sample sizes are much smaller than the current sample sizes. The hyperparameters of the prior distributions have a strong effect on the sample sizes. We discuss methods to deal with this. Even at the mandal (sub-district) level the sample size can almost always be reduced substantially. Clearly SHC over-sampled, and here we show how to reduce the effort while still providing information required for decision-making. R scripts for SSD are provided as supplementary material.

At the start of new operation, a production process is commonly unstable. Then, a process condition is into being stable gradually over time. For the purpose of shifting to mass production, the stability in a process should be evaluated. This paper proposes a method of evaluating the process stability based on Bayesian statistics and information theory. In Bayesian statistics, the knowledge for a process is renewed by deriving posterior distribution based on current prior distribution and observations. Here, equivalency between prior and posterior distributions could be considered to be a criterion for the stability in a process. It is needed to define difference between prior and posterior distributions in order to evaluate the equivalency between prior and posterior distributions. We first formulate the relation between the prior distribution and the posterior distribution for process mean using Bayesian statistics. Secondly, we evaluate the difference between the both distributions based on Kullback‐Leibler (K‐L) divergence in information theory. Finally, some numerical examples in the method of evaluating the stability in process mean are illustrated. Also, we discuss about a decision rule for the stability in process mean through the numerical investigation.

Researchers in the fields of neural networks, statistics, machine learning, and artificial intelligence have followed three basic approaches to developing new pattern classifiers. Probability Density Function (PDF) classifiers include Gaussian and Gaussian Mixture classifiers which estimate distributions or densities of input features separately for each class. Posterior probability classifiers include multilayer perceptron neural networks with sigmoid nonlinearities and radial basis function networks. These classifiers estimate minimum-error Bayesian a posteriori probabilities (hereafter referred to as posterior probabilities) simultaneously for all classes. Boundary forming classifiers include hard-limiting single-layer perceptrons, hypersphere classifiers, and nearest neighbor classifiers. These classifiers have binary indicator outputs which form decision regions that specify the class of any input pattern. Posterior probability and boundary-forming classifiers are trained using discriminant training. All training data is used simultaneously to estimate Bayesian posterior probabilities or minimize overall classification error rates. PDF classifiers are trained using maximum likelihood approaches which individually model class distributions without regard to overall classification performance. Analytic results are presented which demonstrate that many neural network classifiers can accurately estimate posterior probabilities and that these neural network classifiers can sometimes provide lower error rates than PDF classifiers using the same number of trainable parameters. Experiments also demonstrate how interpretation of network outputs as posterior probabilities makes it possible to estimate the confidence of a classification decision, compensate for differences in class prior probabilities between test and training data, and combine outputs of multiple classifiers over time for speech recognition.

In this study, the relationship between nonlinear model properties and inverse problem solutions is analyzed using a numerical technique based on the inverse problem theory formulated by Mosegaard and Tarantola. According to this theory, the inverse problem and solution are defined via convolution and conjunction of probability density functions (PDFs) that represent stochastic information obtained from the model, observations, and prior knowledge in a joint multidimensional space. This theory provides an explicit analysis of the nonlinear model function, together with information about uncertainties in the model, observations, and prior knowledge through construction of the joint probability density, from which marginal solution functions can then be evaluated. The numerical analysis technique derived from the theory computes the component PDFs in discretized form via a combination of function mapping on a discrete grid in the model and observation phase space and Monte Carlo sampling from known parametric distributions. The efficacy of the numerical analysis technique is demonstrated through its application to two well-known simplified models of atmospheric physics: damped oscillations and Lorenz’s three-component model of dry cellular convection. The major findings of this study include the following: (i) Use of a nonmonotonic forward model in the inverse problem gives rise to the potential for a multimodal posterior PDF, the realization of which depends on the information content of the observations and on observation and model uncertainties. (ii) The cumulative effect of observations over time, space, or both could render the final posterior PDF unimodal, even with the nonmonotonic forward model. (iii) A greater number of independent observations are needed to constrain the solution in the case of a nonmonotonic nonlinear model than for a monotonic nonlinear or linear forward model for a given number of degrees of freedom in control parameter space. (iv) A nonlinear monotonic forward model gives rise to a skewed unimodal posterior PDF, implying a well-posed maximum likelihood inverse problem. (v) The presence of model error greatly increases the possibility of capturing multiple modes in the posterior PDF with the nonmonotonic nonlinear model. (vi) In the case of a nonlinear forward model, use of a Gaussian approximation for the prior update has a similar effect to an increase in model error, which indicates there is the potential to produce a biased mean central estimate even when observations and model are unbiased.

Distributional properties for the generalized p-value for the Behrens–Fisher problem

Best estimate plus uncertainty for the safety assessment of nuclear power plant transient requires, among others, estimating the probability density function (PDF) of physical model parameters in thermal-hydraulic system codes. In that context, Bayesian calibration based on experimental data from separate-effect test facilities are increasingly popular to inform the PDF of a single thermal-hydraulic phenomenon. These calibrations are, however, time intensive, especially when considering multiple time-dependent outputs. Calibrating on many tests with different boundary conditions and potentially different phenomena to derive PDFs applicable to complex transients appears intractable, even using hierarchical modeling. In this paper, we start investigating this problem by considering a set of Flooding Experiments with Blocked Arrays reflood tests with different boundary conditions. We use TRACE v5.0p3 to model time- and space-dependent temperature profiles, pressure drops, and liquid carry-over. Global sensitivity analysis helps screen out noninfluential parameters and gain a detailed understanding of the modeled physics of reflood. The analysis shows that, for all tests, the outputs were sensitive to a similar set of influential model parameters. In turn, Bayesian calibration yields similar posterior PDFs for the influential parameters, and forward propagation of these posterior PDFs yields similar confidence intervals. As such, the information of the investigated tests can well be represented by a unique posterior PDF. Such simplifications, although not general, are welcome to help manage the intensive calibration effort necessary for dealing with complex thermal-hydraulic transients encountered in nuclear power plants.

In this chapter, the CIP is formulated in a statistical framework. Byusing the Bayesian inference theory, we cast the CIP asaninformation transfer problem, in which the prior information andtheinformation transferred from state observations are combinedtoreduce uncertainty in the estimated parameters. The priorinformationis modeled using a probability density function (PDF)called the priorPDF and the inverse solution is also a PDF known asthe posteriorPDF. Because it is a PDF, the inverse solution is alwaysexistent andunique but with uncertainty. When the posterior PDF is ina relativelysimple form, point estimates of the unknown parameterscan bereadily obtained by solving an optimization problem, just as wehavedone in the deterministic framework. When the posterior PDFhas acomplex multimodal shape, however, the non-uniquenessandinstability issues associated with the inverse solution arise again.Forsuch cases, Monte Carlo sampling methods provide powerfultoolsfor learning the posterior PDFs without requiring theiractualfunctional forms be known. Two popular Markov Chain MonteCarlo(MCMC) algorithms are introduced. The application of MCMCforinverse solution and global optimization is also discussed.