Bootstrap Confidence Sets and Hypothesis Tests

Several approaches to hypothesis testing for coefficients in least absolute value regression are compared using a Monte Carlo simulation: likelihood ratio test, Lagrange multiplier test, and three versions of the bootstrap hypothesis test. Factors considered that might influence test performance include the disturbance distribution, the type of independent variable, and the sample size. Overall, the likelihood ratio and the bootstrap tests perform best, with the likelihood ratio test being marginally more powerful. Least absolute value tests are also compared to the standard t test and three versions of the bootstrapped t test for least squares regression.

Chapter 1. Role of statistics and data analysis. 1.1 Introduction. 1.2 Case studies. 1.3 Data. 1.4 Samples versus the population, some notation. 1.5 Vector and matrix notation. 1.6 Frequency distributions and histograms 1.7 The distribution as a model. 1.8 Sample moments. 1.9 Normal (Gaussian) distribution. 1.10 Exploratory data analysis. 1.11 Estimation. 1.12 Bias. 1.13 Causes of variance. 1.14 About data. 1.15 Reasons to conduct statistically based studies. 1.16 Data mining. 1.17 Modeling. 1.18 Transformations. 1.19 Statistical concepts. 1.20 Statistics paradigms. 1.21 Summary. 1.22 Exercises. Chapter 2. Modeling concepts. 2.1 Introduction. 2.2 Why construct a model? 2.3 What does a statistical model do? 2.4 Steps in modeling. 2.5 Is a model a unique solution to a problem? 2.6 Model assumptions. 2.7 Designed experiments. 2.8 Replication. 2.9 Summary. 2.10 Exercises. Chapter 3. Estimation and hypothesis testing on means and other statistics. 3.1 Introduction. 3.2 Independence of observations. 3.3 The Central Limit Theorem. 3.4 Sampling distributions. 3.4.1 t-distribution. 3.5 Confidence interval estimate on a mean. 3.6 Confidence interval on the difference between means. 3.7 Hypothesis testing on means. 3.8 Bayesian hypothesis testing. 3.9 Nonparametric hypothesis testing. 3.10 Bootstrap hypothesis testing on means. 3.11 Testing multiple means via analysis of variance. 3.12 Multiple comparisons of means. 3.13 Nonparametric ANOVA. 3.14 Paired data. 3.15 Kolmogorov-Smirnov goodness-of-fit test. 3.16 Comments on hypothesis testing. 3.17 Summary. 3.18 Exercises. Chapter 4. Regression. 4.1 Introduction. 4.2 Pittsburgh coal quality case study. 4.3 Correlation and covariance. 4.4 Simple linear regression. 4.5 Multiple regression. 4.6 Other regression procedures. 4.7 Nonlinear models. 4.8 Summary. 4.9 Exercises. Chapter 5. Time series. 5.1 Introduction. 5.2 Time Domain. 5.3 Frequency Domain. 5.4 Wavelets. 5.5 Summary. 5.6 Exercises. Chapter 6. Spatial statistics. 6.1 Introduction. 6.2 Data. 6.3 Three-dimensional data visualization. 6.4 Spatial association. 6.5 The effect of trend. 6.6 Semivariogram models. 6.7 Kriging. 6.8 Space-time models. 6.9 Summary. 6.10 Exercises. Chapter 7. Multivariate analysis. 7.1 Introduction. 7.2 Multivariate graphics. 7.3 Principal component analysis. 7.4 Factor analysis. 7.5 Cluster analysis. 7.6 Multidimensional scaling. 7.7 Discriminant analysis. 7.8 Tree based modeling. 7.9 Summary. 7.10 Exercises. Chapter 8. Discrete data analysis and point processes. 8.1 Introduction. 8.2 Discrete process and distributions. 8.3 Point processes. 8.4 Lattice data and models. 8.5 Proportions. 8.6 Contingency tables. 8.7 Generalized linear models. 8.8 Summary. 8.9 Exercises. Chapter 9 Design of experiments. 9.1 Introduction. 9.2 Sampling designs. 9.3 Design of experiments. 9.4 Comments on field studies and design. 9.5 Missing data. 9.6 Summary. 9.7 Exercises. Chapter 10 Directional data. 10.1 Introduction. 10.2 Circular data. 10.3 Spherical data. 10.4 Summary. 10.5 Exercises.

The bootstrap is a very convenient and appealing tool for statistical analysis, however, theoretical and/or empirical confirmation should be made of its suitability for the problem at hand. Also, it is important to know the relative performance of the bootstrap versus other existing methods. A general theory for the bootstrap distribution and variance estimation for a given statistic is presented in this chapter. The more delicate problem of constructing bootstrap confidence sets and hypothesis tests will be treated in the next chapter.

Two guidelines for nonparametric bootstrap hypothesis testing are highlighted. The first recommends that resampling be done in a way that reflects the null hypothesis, even when the true hypothesis is distant from the null. The second guideline argues that bootstrap hypothesis tests should employ methods that are already recognized as having good features in the closely related problem of confidence interval construction. Violation of the first guideline can seriously reduce the power of a test. Sometimes this reduction is spectacular, since it is most serious when the null hypothesis is grossly in error. The second guideline is of some importance when the conclusion of a test is equivocal. It has no direct bearing on power, but improves the level accuracy of a test.

Cumulative Sum Curves and Their Prediction Limits

Bootstrap hypothesis testing for some common statistical problems: A critical evaluation of size and power properties

A method of structural damage identification using harmonic excitation force is presented. It considers the effects of both measurement and modelling errors in the baseline finite element model. Damage that accompanies changes in structural parameters can be estimated for a damaged structure from the change between measured vibration responses and ones calculated from the analytical model of the intact structure. In practice, modelling errors exist in the analytical model due to material and geometric uncertainties and a reduction in the degrees of freedom as well as measurement errors, making identification difficult. To surmount these problems, bootstrap hypothesis testing, which enables statistical judgment without information about these errors, was introduced. The method was validated by numerical simulation using a three-dimensional frame structure and real vibration data for a three-storey steel frame structure. Copyright © 2005 John Wiley & Sons, Ltd.

Re-sampling based statistical tests are known to be computationally heavy, but reliable when small sample sizes are available. Despite their nice theoretical properties not much effort has been put to make them efficient. Computationally efficient method for calculating permutation-based p-values for the Pearson correlation coefficient and two independent samples t-test are proposed. The method is general and can be applied to other similar two sample mean or two mean vectors cases.

Residual bootstrap tests in linear models with many regressors

In recent years, empirical Bayesian (EB) inference has become an attractive approach for estimation in parametric models arising in a variety of real‐life problems, especially in complex and high‐dimensional scientific applications. However, compared to the relative abundance of available general methods for computing point estimators in the EB framework, the construction of confidence sets and hypothesis tests with good theoretical properties remains difficult and problem specific. Motivated by the Universal Inference framework, we propose a general and universal method, based on holdout likelihood ratios, and utilizing the hierarchical structure of the specified Bayesian model for constructing confidence sets and hypothesis tests that are finite sample valid. We illustrate our method through a range of numerical studies and real data applications, which demonstrate that the approach is able to generate useful and meaningful inferential statements in the relevant contexts.

The paper studies a problem of constructing simultaneous likelihood-based confidence sets. We consider a simultaneous multiplier bootstrap procedure for estimating the quantiles of the joint distribution of the likelihood ratio statistics, and for adjusting the confidence level for multiplicity. Theoretical results state the bootstrap validity in the following setting: the sample size n is fixed, the maximal parameter dimension p_max and the number of considered parametric models K are s.t. (log?K )^12 p_max^3/n is small. We also consider the situation when the parametric models are misspecified. If the models' misspecification is significant, then the bootstrap critical values exceed the true ones and the simultaneous bootstrap confidence set becomes conservative. Numerical experiments for local constant and local quadratic regressions illustrate the theoretical results.

Air pressure readings and their variations are commonly used to make inferences about storm activity. More precisely, it is assumed that the variation of annual and seasonal statistics of several pressure-based proxies describes changes in the past storm climate qualitatively, an assumption that has yet to be proven. A systematic evaluation of the informational content of five pressure-based proxies for storm activity based on single-station observations of air pressure is presented. The number of deep lows, lower percentiles of pressure, the frequency of absolute pressure tendencies above certain thresholds, as well as mean values and high percentiles of absolute pressure tendencies is examined. Such an evaluation needs long and homogeneous records of wind speed, something that is not available from observations. Consequently, the proxies are examined by using datasets of ground-level wind speeds and air pressure from the NCEP-driven and spectrally nudged regional model, REMO. The proxies are gauged against the 95th and 99th percentile time series of ground-level wind speeds to quantify the relation between pressure-based proxies and storminess. These analyses rely on bootstrap and binomial hypothesis testing. The analyses of single-station-based proxies indicate that the proxies are generally linearly linked to storm activity, and that absolute pressure tendencies have the highest informational content. Further, it is investigated as to whether the proxies have the potential for describing storminess over larger areas, also with regard to surface conditions. It is found that absolute pressure tendencies have improved informational value when describing storm activity over larger areas, while low pressure readings do not show improved informational value.

SUMMARY Confidence intervals for the relative frequency of responding cells in limiting dilution assays (LDAs) have not been examined closely. Generally, confidence intervals are calculated by using the normal approximation, sometimes preceded by a log transform. We evaluate and compare with analytical approaches: the normal approximation, the log transform, and two binomial methods-the quadratic approximation and a modification of the Clopper-Pearson exact method. We evaluate these confidence interval methods for use with the maximum likelihood and jackknife estimators of the relative frequency. Our results show that confidence interval construction with maximum likelihood point estimates is more accurate than construction with jackknife estimates. When using maximum likelihood estimates, the normal approximation produces acceptable two-sided confidence intervals, with the smallest length, at a = .05. At a = .01, the normal approximation is anticonservative. In all cases, the normal approximation is unable to produce adequate one-sided confidence intervals. The log transform and both binomial confidence interval methods are shown to be generally superior to

This paper introduces the “piggyback bootstrap.” Like the weighted bootstrap, this bootstrap procedure can be used to generate random draws that approximate the joint sampling distribution of the parametric and nonparametric maximum likelihood estimators in various semiparametric models, but the dimension of the maximization problem for each bootstrapped likelihood is smaller. This reduction results in significant computational savings in comparison to the weighted bootstrap. The procedure can be stated quite simply. First obtain a valid random draw for the parametric component of the model. Then take the draw for the nonparametric component to be the maximizer of the weighted bootstrap likelihood with the parametric component fixed at the parametric draw. We prove the procedure is valid for a class of semiparametric models that includes frailty regression models airsing in survival analysis and biased sampling models that have application to vaccine efficacy trials. Bootstrap confidence sets from the piggyback, and weighted bootstraps are compared for biased sampling data from simulated vaccine efficacy trials.

If it hasn’t failed, does it work? On ‘the worst we can expect’ from observational trial results, with reference to airway management devices