Stability evaluation in process mean using Bayesian statistics and information theory

A crucial decision in designing a spatial sample for soil survey is the number of sampling locations required to answer, with sufficient accuracy and precision, the questions posed by decision makers at different levels of geographic aggregation. In the Indian Soil Health Card (SHC) scheme, many thousands of locations are sampled per district. In this paper the SHC data are used to estimate the mean of a soil property within a defined study area, e.g., a district, or the areal fraction of the study area where some condition is satisfied, e.g., exceedence of a critical level. The central question is whether this large sample size is needed for this aim. The sample size required for a given maximum length of a confidence interval can be computed with formulas from classical sampling theory, using a prior estimate of the variance of the property of interest within the study area. Similarly, for the areal fraction a prior estimate of this fraction is required. In practice we are uncertain about these prior estimates, and our uncertainty is not accounted for in classical sample size determination (SSD). This deficiency can be overcome with a Bayesian approach, in which the prior estimate of the variance or areal fraction is replaced by a prior distribution. Once new data from the sample are available, this prior distribution is updated to a posterior distribution using Bayes’ rule. The apparent problem with a Bayesian approach prior to a sampling campaign is that the data are not yet available. This dilemma can be solved by computing, for a given sample size, the predictive distribution of the data, given a prior distribution on the population and design parameter. Thus we do not have a single vector with data values, but a finite or infinite set of possible data vectors. As a consequence, we have as many posterior distribution functions as we have data vectors. This leads to a probability distribution of lengths or coverages of Bayesian credible intervals, from which various criteria for SSD can be derived. Besides the fully Bayesian approach, a mixed Bayesian-likelihood approach for SSD is available. This is of interest when, after the data have been collected, we prefer to estimate the mean from these data only, using the frequentist approach, ignoring the prior distribution. The fully Bayesian and mixed Bayesian-likelihood approach are illustrated for estimating the mean of log-transformed Zn and the areal fraction with Zn-deficiency, defined as Zn concentration <0.9 mg kg −1, in the thirteen districts of Andhra Pradesh state. The SHC data from 2015–2017 are used to derive prior distributions. For all districts the Bayesian and mixed Bayesian-likelihood sample sizes are much smaller than the current sample sizes. The hyperparameters of the prior distributions have a strong effect on the sample sizes. We discuss methods to deal with this. Even at the mandal (sub-district) level the sample size can almost always be reduced substantially. Clearly SHC over-sampled, and here we show how to reduce the effort while still providing information required for decision-making. R scripts for SSD are provided as supplementary material.

For more than three decades, Deborah Mayo has defended severe testing, a framework for evaluating the strength of scientific evidence that provides philosophical foundations for (variants of) classical statistical methods. In her newest book, Statistical Inference as Severe Testing, Mayo (1) extends and refines her views on severe testing, (2) takes aim at researchers who claim that the reproducibility crisis is a result of the use of classical statistics, and (3) exposes some contemporary Bayesian methods as lacking philosophical foundations. The book is engaging, sometimes funny, and often insightful. After reviewing the basics of severe testing, I summarize three lessons from Mayo's book that are valuable for philosophers and scientists alike. I then discuss one way in which I think Mayo's arguments could be tightened.Mayo argues that one has evidence for a hypothesis to the extent the hypothesis has passed a severe test. More precisely, she argues that “for data to warrant a hypothesis H requires not just that (S-1) H agrees with the data (H passes the test), but also (S-2) with high probability, H would not have passed the test so well, were H false” (92). Mayo's two criteria are inspired by the Neyman-Pearsonian maxim that good statistical tests minimize Type 1 and Type 2 errors, but they are novel in at least two ways. First, Mayo's criteria are applicable outside of statistical contexts. For example, let H be the hypothesis that the Cleveland Browns played an excellent team last weekend, and suppose our data is that the Cleveland Browns lost their game. If the Cleveland Browns lose to all opponents regardless of ability, then our data fails to satisfy S-2 and so does not severely test H. Since S-1 and S-2 are applicable in both statistical and nonquantitative contexts, Mayo's criteria unify the logic of statistical testing with qualitative logics of evidence.Second, severity is a function of data, unlike size and power, the two core quantities of orthodox Neyman-Pearsonian methodology. For this reason, Mayo's two criteria closely resemble Nozick's adherence and sensitivity conditions when those conditions are relativized to “methods” (Nozick 1981).1Mayo draws several important lessons from this seemingly simple, two-pronged framework. Here, I discuss three that recur throughout the newest book. First, Mayo carefully distinguishes between statistical hypotheses (e.g., about the frequency of a trait in a population) and “substantive” scientific hypotheses about, for example, causation or the existence of a theoretical entity (84, 93). Conflating the two can lead one to mistakenly infer that a substantive scientific hypothesis has been severely tested, when in fact the evidence against the hypothesis is stronger than that for it.As a vivid example, Mayo discusses a series of psychological experiments used to defend the hypothesis H that heterosexual “men's implicit self-esteem is lower when a [female] partner succeeds than when a [female] partner fails” (101). Mayo carefully distinguishes four statistical hypotheses that are plausibly consequences of ¬ H and then argues that the data were sufficient to reject only one of the four; call it ¬ H 1. Mayo concludes that the experimental data failed to fit H, as researchers could not reject three statistical hypotheses that were compatible with ¬ H. Further, Mayo argues that the statistical evidence against ¬ H 1 (i.e., the purported evidence for ¬¬ H = H ) was perfectly compatible with the falsity of the substantive hypothesis H. So the data also failed to satisfy S-2.Second, Mayo argues that, whereas the severe tester has a unified account of testing both statistical hypotheses and modeling assumptions, likelihoodists and Bayesians lack a cogent story about when to reject a bad statistical model (298–320). Some likelihoodists, for example, argue that measures of evidential strength must be comparative and that the likelihood ratio P (E|H 0)/P (E|H 1) quantifies the degree to which evidence E fits a hypothesis H 0 better than H 1 (Edwards 1984; Royall 1997). It is easy to construct cases in which such a likelihood ratio is enormous, because the data fit H 0 much better than H 1, and yet the data fit neither statistical hypothesis well. If H 0 and H 1 are the only statistical hypotheses under investigation, one might erroneously conclude that one has good evidence for H 0. The right conclusion, according to the severe tester, is to recognize that the two hypotheses do not exhaust logical space and to reject that underlying modeling assumption instead.Naive Bayesian reasoning falls victim to the same criticism. According to some Bayesians, experimenters ought to assign prior probabilities to a fixed set of hypotheses H and then update their degrees of belief by conditionalization. Again, suppose the data fit all hypotheses in H poorly but fit one H ∈ H much better than its rivals. Then a naive Bayesian's posterior degree of belief in H might be nearly one. Again, the right conclusion, according to the severe tester, is to consider more hypotheses. But orthodox Bayesianism is silent about when to do that.Finally, Mayo argues that several novel Bayesian approaches to statistical inference lack foundations (400–415). These parts of the book will be of special interest to philosophers who are unfamiliar with the substantial differences between Bayesianism in theory and in practice. According to textbook philosophical presentations, prior and posterior probabilities represent an experimenter's beliefs. However, some contemporary Bayesians have given up on the myth that the infinitely precise densities that appear in journal articles are appropriate idealizations of any person's beliefs. Those Bayesians, however, disagree about what justifies the choice of prior (e.g., transparent communication vs. frequentist coverage properties) and the use of further Bayesian mathematical machinery.Mayo levels several different criticisms against these contemporary Bayesian approaches, but I think all share the same underlying philosophical problem. Rationality, so says standard decision theory, requires one to maximize expected utility relative to one own's degrees of belief. So some contemporary “Bayesian” methods require a new theory of rationality to justify the behavior of an experimenter whose decisions employ posterior probabilities that do not even approximately represent her (or perhaps anyone's) beliefs.There are other important lessons scattered through Mayo's engaging and fairly accessible book. That leads me to my main criticism: accessibility sometimes comes at the cost of precision. I focus on one example that is directly relevant to Mayo's analysis of the causes of the replicability crisis, a central topic of the book.Mayo repeatedly suggests that a hypothesis is not severely tested if it is the result of “data-dredging.” Data-dredging is one of several pejorative terms (including p-hacking) that some statistical reformers use, and it refers to a loosely defined set of purportedly objectionable practices for finding associations in data. Often, “data-dredging” describes the practice of subjecting a single data set to many (sometimes hundreds or thousands of) statistical tests in an attempt to find a statistically significant result that is publishable.Critics of data-dredging allege that, because there is always a chance of finding a spurious association between two variables, one should be wary of “dredged” hypotheses. For severe testers, that default skepticism is justifiable: if an experiment is not designed to test a hypothesis, it is unlikely to be a severe test by chance. However, intuitions about data-dredging depend upon how the practice is described. Searching through data for patterns sounds exactly like what scientists should do!Mayo illustrates the dangers of data-dredging via several thought experiments. For example, she imagines a pharmaceutical company that hires scientists to find the benefits of a recently developed drug (267–68). When the hired researchers find no evidence that the drug is salutary in any of ways the manufacturer had hoped, they mine their data to find some salutary effect. Sure enough, they find the drug has an “impressive benefit on factor B” and begin marketing the drug with that promise. Mayo concludes that the hired scientists fail to subject the hypothesis to a severe test because they would have found an effect no matter what.Unfortunately, Mayo never explicitly applies her two criteria for severity to this example. Which hypothesis exactly is being tested? Here are two candidates. Let H 1 be the hypothesis “The drug has some salutary effect” and let H 2 be the hypothesis “The drug has the specific salutary effect found on B.” Even if the drug manufacturer is guaranteed to find some effect (i.e., H 1 is not severely tested), it does not follow that the manufacturer lacks evidence that the drug has the specific effect found by mining the data (i.e., it does not follow that H 2 is not severely tested). When I grade student logic exams, I am guaranteed to find that some student scored highest, but I do not thereby lack evidence that Roshan scored best when I tally her score and see it is highest. Similarly, cigarette manufacturers cannot avoid liability for causing lung cancer by (1) collecting enormous amounts of data, (2) investigating the relationship between cigarette smoking and hundreds of health problems, and then (3) claiming they were destined to find some statistically significant effect or another.2Some accusations of data-dredging, I claim, are instances of a fallacy that Mayo typically distinguishes for her readers with great care: the conflation of the pre-experimental reliability of a test with the postexperimental evidence provided by a particular data set.3 The history of science is replete with examples of unlikely, happy discoveries, where the data unequivocally support a hypothesis that scientists did not intend to test. In accessible, vivid prose, Mayo illustrates why one should typically be wary of data-dredging, but readers will be disappointed if they wish to learn how Mayo's two criteria can be used to distinguish between (a) hypotheses that are illicitly “dredged” and (b) hypotheses that are supported by one's data, despite being formulated postexperiment.Quibbles aside, Mayo's newest book is a yet another deep and engaging contribution to the philosophy of statistics and philosophy of science. Most importantly, it shows how foundational philosophical debates matter to pressing issues in science policy.

A trending approach for analysis in critical care research: Bayesian statistics.

In Bayesian phylogenetics, confidence in evolutionary relationships is expressed as posterior probability--the probability that a tree or clade is true given the data, evolutionary model, and prior assumptions about model parameters. Model parameters, such as branch lengths, are never known in advance; Bayesian methods incorporate this uncertainty by integrating over a range of plausible values given an assumed prior probability distribution for each parameter. Little is known about the effects of integrating over branch length uncertainty on posterior probabilities when different priors are assumed. Here, we show that integrating over uncertainty using a wide range of typical prior assumptions strongly affects posterior probabilities, causing them to deviate from those that would be inferred if branch lengths were known in advance; only when there is no uncertainty to integrate over does the average posterior probability of a group of trees accurately predict the proportion of correct trees in the group. The pattern of branch lengths on the true tree determines whether integrating over uncertainty pushes posterior probabilities upward or downward. The magnitude of the effect depends on the specific prior distributions used and the length of the sequences analyzed. Under realistic conditions, however, even extraordinarily long sequences are not enough to prevent frequent inference of incorrect clades with strong support. We found that across a range of conditions, diffuse priors--either flat or exponential distributions with moderate to large means--provide more reliable inferences than small-mean exponential priors. An empirical Bayes approach that fixes branch lengths at their maximum likelihood estimates yields posterior probabilities that more closely match those that would be inferred if the true branch lengths were known in advance and reduces the rate of strongly supported false inferences compared with fully Bayesian integration.

CHAPTER 8 - Decision Theoretic and Bayesian Approach in Testing and Estimation

Mathematical theory of Bayesian statistics where all models are wrong

We discuss the problem of model criticism, with emphasis on developing summary diagnostic measures. We approach model criticism by identifying possible troublesome features of the currently entertained model, embedding the model in an elaborated model, and measuring the value of elaborating. This requires three elements: a model elaboration, a prior distribution, and a utility function. Each triplet generates a different diagnostic measure. We focus primarily on the measure given by a Kullback—Leibler divergence between the marginal prior and posterior distributions on the elaboration parameter. We also develop a linearized version of this diagnostic and use it to show that our procedure is related to other tools commonly used for model diagnostics, such as Bayes factors and the score function. One attraction of this approach is that it allows model criticism to be performed jointly with parameter inference and prediction. Also, this diagnostic approach aims at maintaining an exploratory nature t...

Presumptive drug testing is commonly used in both the clinical and forensic fields to allow rapid identification of the presence and/or usage of drugs. Because the tests generally have a high sensitivity and specificity (often &gt;90%), then a positive test result may be taken to mean there is a high probability that a targeted drug is present. This assumption is, however, incorrect. This paper demonstrates how, in order to assess the positive predictive value (PPV) of a test, it is necessary to take into account, along with the sensitivity and specificity of the test, the prevalence of the drug in the population being investigated. We demonstrate how an alternative, Bayesian approach to assessing the posterior probability of a drug being present mimics the conventional calculation of PPVs but, because a Bayesian approach requires case‐specific prior probabilities, the posterior probabilities are more meaningful than PPV in any one specific case. The effectiveness of presumptive test results in cases such as drink‐spiking, drug‐driving, testing of drugs during seizures and the confirmation of initial presumptive test results is explored. In order to exploit the potential of presumptive drug testing, it is important that the prevalence of the targeted drugs in relevant populations is understood but, more importantly, it is important to consider using a Bayesian approach in order to tailor results to the specific individual or drug batch being tested.This article is categorized under: Toxicology &gt; Analytical Toxicology Jurisprudence and Regulatory Oversight &gt; Expert Evidence and Narrative

Knowledge selection plays an important role in knowledge-grounded dialogue, which is a challenging task to generate more informative responses by leveraging external knowledge. Recently, latent variable models have been proposed to deal with the diversity of knowledge selection by using both prior and posterior distributions over knowledge and achieve promising performance. However, these models suffer from a huge gap between prior and posterior knowledge selection. Firstly, the prior selection module may not learn to select knowledge properly because of lacking the necessary posterior information. Secondly, latent variable models suffer from the exposure bias that dialogue generation is based on the knowledge selected from the posterior distribution at training but from the prior distribution at inference. Here, we deal with these issues on two aspects: (1) We enhance the prior selection module with the necessary posterior information obtained from the specially designed Posterior Information Prediction Module (PIPM); (2) We propose a Knowledge Distillation Based Training Strategy (KDBTS) to train the decoder with the knowledge selected from the prior distribution, removing the exposure bias of knowledge selection. Experimental results on two knowledge-grounded dialogue datasets show that both PIPM and KDBTS achieve performance improvement over the state-of-theart latent variable model and their combination shows further improvement.

High uncertainties arias through the characterization of soil parameters because of the lack of data obtained from geotechnical reports. Reducing these uncertainties may improve the characteristic values of soil parameters. This research aims to probabilistically characterize a soil's cohesion parameter in Nasiriyah. The Bayesian approach has been applied to soil data obtained through a project in Nasiriyah. The soil at the site is classified as lean clay, and the soil cohesion has been evaluated using two Bayesian methods: the ordinary, normal distribution method (OND) and the Marcove Chain Monte Carlo-based Bayesian approach (MCMC) method. The previous knowledge utilized in the Bayesian approach was based on 20 boreholes, and the subjective probability approach has functioned in the prior probability distribution. The OND method deduced a mean value of cohesion of (195.9 kPa) and a standard deviation of (14.68 kPa), (COV) 7.49%. It was noted that the probability distribution has a more significant effect than the previous distribution on the posterior distribution. The MCMC method summarized the probabilistic description of the soil characteristic, through which it reached the mean and the subsequent standard deviation (167.49) kPa (109.8) kPa, respectively, and the coefficient of Variation (COV) was 65.6%. It is considered the most appropriate and common method, especially in high-dimensional data when the results are not well known because it can provide a probabilistic value for the not well-known data.

The aim of this study was to explore whether Bayesian reasoning can be applied to therapeutic questions in a way that is similar to its application in diagnostics. A clinically relevant, therapeutic question was formulated in accordance with Bayesian reasoning for the clinical management of patients with newly diagnosed rheumatoid arthritis (RA). Prior probability estimates of response to drug treatment (methotrexate, MTX) were obtained from the literature. As a marker of treatment response, changes in the Health Assessment Questionnaire (HAQ) scores were assessed after three months of treatment. Likelihood ratios for this marker were calculated on the basis of data from a clinical registry, using changes in the Disease Activity Score (DAS) as gold standard. Using Bayes' theorem, prior probability and likelihood ratios were combined to estimate posterior probabilities of treatment response in individual patients. On the basis of the literature, the prior probability of response of RA patients to MTX was estimated 45 percent. At 3 months follow-up, this probability increased to 80 percent or decreased to 23 percent, depending on the changes that were observed in Health Assessment Questionnaire scores. Bayesian reasoning can be applied to therapeutic issues in a way that is conceptually fully compatible with its use in diagnostics. As such, it can be used to bridge the gap between aggregate data and individual patient management.

Medical image segmentation using scalable functional variational Bayesian neural networks with Gaussian processes

Approximated sensitivity analysis in posterior predictive distribution

This textbook explains the basic ideas of subjective probability and shows how subjective probabilities must obey the usual rules of probability to ensure coherency. It defines the likelihood function, prior distributions and posterior distributions. It explains how posterior distributions are the basis for inference and explores their basic properties. Various methods of specifying prior distributions are considered, with special emphasis on subject-matter considerations and exchange ability. The regression model is examined to show how analytical methods may fail in the derivation of marginal posterior distributions. The remainder of the book is concerned with applications of the theory to important models that are used in economics, political science, biostatistics and other applied fields. New to the second edition is a chapter on semiparametric regression and new sections on the ordinal probit, item response, factor analysis, ARCH-GARCH and stochastic volatility models. The new edition also emphasizes the R programming language.

This chapter is devoted to some basic ideas on the Bayesian approach to statistical inference, where the parameter is treated as a random variable, which is assigned a distribution. This distribution – the prior distribution – represents the prior knowledge about the parameter before observing the data. Once the data is observed, the inference about parameter is drawn from the posterior distribution – the conditional distribution of parameter given the data. The term “Bayesian” comes from the well-known Bayes theorem, which is a formula for computing the posterior probabilities.