Robust group inference for ultrahigh-dimensional linear regression models

The research on statistical inference after data-driven model selection can be traced as far back as Koopmans (1949). The intensive research on modern model selection methods for high-dimensional data over the past three decades revived the interest in statistical inference after model selection. In recent years, there has been a surge of articles on statistical inference after model selection and now a rather vast literature exists on this topic. Our manuscript aims at presenting a holistic review of post-model-selection inference in linear regression models, while also incorporating perspectives from high-dimensional inference in these models. We first give a simulated example motivating the necessity for valid statistical inference after model selection. We then provide theoretical insights explaining the phenomena observed in the example. This is done through a literature survey on the post-selection sampling distribution of regression parameter estimators and properties of coverage probabilities of naïve confidence intervals. Categorized according to two types of estimation targets, namely the population- and projection-based regression coefficients, we present a review of recent uncertainty assessment methods. We also discuss possible pros and cons for the confidence intervals constructed by different methods.

The linear regression model is widely used in empirical work in Economics, Statistics, and many other disciplines. Researchers often include many covariates in their linear model specification in an attempt to control for confounders. We give inference methods that allow for many covariates and heteroskedasticity. Our results are obtained using high-dimensional approximations, where the number of included covariates are allowed to grow as fast as the sample size. We find that all of the usual versions of Eicker-White heteroskedasticity consistent standard error estimators for linear models are inconsistent under this asymptotics. We then propose a new heteroskedasticity consistent standard error formula that is fully automatic and robust to both (conditional) heteroskedasticity of unknown form and the inclusion of possibly many covariates. We apply our findings to three settings: parametric linear models with many covariates, linear panel models with many fixed effects, and semiparametric semi-linear models with many technical regressors. Simulation evidence consistent with our theoretical results is also provided. The proposed methods are also illustrated with an empirical application.

ABSTRACTThe linear regression model is widely used in empirical work in economics, statistics, and many other disciplines. Researchers often include many covariates in their linear model specification in an attempt to control for confounders. We give inference methods that allow for many covariates and heteroscedasticity. Our results are obtained using high-dimensional approximations, where the number of included covariates is allowed to grow as fast as the sample size. We find that all of the usual versions of Eicker–White heteroscedasticity consistent standard error estimators for linear models are inconsistent under this asymptotics. We then propose a new heteroscedasticity consistent standard error formula that is fully automatic and robust to both (conditional) heteroscedasticity of unknown form and the inclusion of possibly many covariates. We apply our findings to three settings: parametric linear models with many covariates, linear panel models with many fixed effects, and semiparametric semi-linear models with many technical regressors. Simulation evidence consistent with our theoretical results is provided, and the proposed methods are also illustrated with an empirical application. Supplementary materials for this article are available online.

Robust inference in nonlinear models with mixed identification strength

Bayesian inference for normal regression models, including sensitivity analysis, model comparison and error in variables under non-informative and conjugate prior for the model parameters has received considerable attention in the last decades. From a distributional point of view the results can be extended in several directions. One is by considering a wider class of prior distributions for the parameters of the model. Another, is by considering alternative distributions for the error terms. Usually, the results with non-conjugate priors rely heavily on MCMC methods. On the other hand, many extensions has been obtained by considering the so called dependent elliptical model, which is often used in linear regression analysis to accommodate the kurtosis of the error terms and to accommodate outliers. Bayesian inference with multivariate elliptical models was initially presented in Chu (1973). Posteriorly, Meinhold and Singpurwalla (1989) consider a robustification of Kalman filters by using the multivariate Student-t distribution. This distribution was also used by Zellner (1976), who considered a Bayesian treatment of linear regression models under non-informative prior distributions.

Finding a new drug which is both safe and efficient is an expensive and time-consuming endeavour. In particular, establishing the ‘dose-effect relationship’ – how beneficial a drug is at different dosages – can be challenging. Predicting this curve requires gathering experimental data by exposing and recording how cells respond to various levels of the drug. However, extreme values are often observed at low and high dosages, potentially introducing errors that are hard to correct in the prediction process. Yet, these extreme observations are sometimes genuine so researchers cannot just ignore them. To improve dose-effect estimation, Zhou, Liu, Fang et al. developed a new general-purpose approach. It uses advanced statistical modelling to account for extremes in lab data. This strategy outperformed other methods when dealing with these observations while also providing higher efficiency in data analysis with more uniform data in experiments. To facilitate implementation, Zhou, Liu, Fang et al. set up a user-friendly tool baptized ‘REAP’; this free online resource allows scientists without advanced statistical experience to harness the new approach and to perform dose-effect analysis more easily and accurately. This could boost research across many different disciplines that examine the effects of chemicals on cells.

Estimation in linear regression models with measurement errors subject to single-indexed distortion

In this chapter we consider dependent data but move from the linear models of Chap.8 to general regression models. As in Chap.6, we consider generalized linear models (GLMs) and, more briefly, nonlinear models. We first give an outline of this chapter. In Sect.9.2 we describe three motivating datasets to which we return throughout the chapter. The GLMs discussed in Sect.6.3 can be extended to incorporate dependences in observations on the same unit; as with the linear model, an obvious way to carry out modeling in this case is to introduce unit-specific random effects. Within a GLM a natural approach is for these random effects to be included on the linear predictor scale. The resultant conditional models are known as generalized linear mixed models (GLMMs), and these are introduced in Sect.9.3. In Sects.9.4 and9.5 we describe likelihood and conditional likelihood methods of estimation, with Sect.9.6 devoted to a Bayesian treatment. Section9.7 illustrates some of the flexibility of GLMMs by describing and applying a particular model for spatial dependence. An alternative random effects specification, based on conjugacy, is described in Sect.9.8. An important approach to the modeling and analysis of dependent data that is philosophically different from the random effects formulation is via marginal models and generalized estimating equations (GEE), and these are the subject of Sect.9.9. In Sect.9.10, a second GEE approach is described in which the estimating equations for the mean are supplemented with a second set for the variances/covariances. For GLMMs, extra care must be taken with parameter interpretation, and Sect.9.11 discusses this issue, emphasizing how interpretation differs between conditional and marginal models. In PartII of the book, which focused on independent data, Chap.7 was devoted to models for binary data. For dependent data, models binary data are less well developed, and so we do not devote a complete chapter to their description. However, Sect.9.12 introduces the modeling of dependent binary data, and, subsequently, Sects.9.13 and 9.14 describe conditional (mixed) and marginal models for binary data. Section 9.15 considers how nonlinear models, as defined in Sect.6.10, can be extended to the dependent data case. For such models, many applications concentrate on inference for units, and so the introduction of random effects is again suggested. We refer to the resultant class of models as nonlinear mixed models (NLMMs). Section9.16 considers issues related to the parameterization of the nonlinear model. Inference for nonlinear mixed models via likelihood and Bayes approaches is covered in Sects.9.17 and9.18, while GEE is briefly considered in Sect.9.19. The assessment of assumptions for general regression models is described in Sect.9.20, with concluding comments contained in Sect.9.21. Additional references appear in Sect.9.22.

Heteroskedasticity and spatiotemporal dependence robust inference for linear panel models with fixed effects

This paper reconciles the asymptotic disagreement between Bayesian and frequentist inference in set-identified models by adopting a multiple-prior (robust) Bayesian approach. We propose new tools for Bayesian inference in set-identified models. We show that these tools have a well-defined posterior interpretation in finite samples and are asymptotically valid from the frequentist perspective. The main idea is to construct a prior class that removes the source of the disagreement: the need to specify an unrevisable prior. The corresponding class of posteriors can be summarized by reporting the "posterior lower and upper probabilities" of a given event and/or the "set of posterior means" and the associated "robust credible region". We show that the set of posterior means is a consistent estimator of the true identified set and the robust credible region has the correct frequentist asymptotic coverage for the true identified set if it is convex. Otherwise, the method can be interpreted as providing posterior inference about the convex hull of the identified set. For impulse-response analysis in set-identified Structural Vector Autoregressions, the new tools can be used to overcome or quantify the sensitivity of standard Bayesian inference to the choice of an unrevisable prior.

This paper reconciles the asymptotic disagreement between Bayesian and frequentist inference in set-identified models by adopting a multiple-prior (robust) Bayesian approach. We propose new tools for Bayesian inference in set-identified models and show that they have a well-defined posterior interpretation infinite samples and are asymptotically valid from the frequentist perspective. The main idea is to construct a prior class that removes the source of the disagreement: the need to specify an unrevisable prior for the structural parameter given the reduced-form parameter. The corresponding class of posteriors can be summarized by reporting the 'posterior lower and upper probabilities' of a given event and/or the 'set of posterior means' and the associated 'robust credible region'. We show that the set of posterior means is a consistent estimator of the true identi?ed set and the robust credible region has the correct frequentist asymptotic coverage for the true identified set if it is convex. Otherwise, the method provides posterior inference about the convex hull of the identified set. For impulse-response analysis in set-identified Structural Vector Autoregressions, the new tools can be used to overcome or quantify the sensitivity of standard Bayesian inference to the choice of an unrevisable prior.

Pharmacokinetic-Pharmacodynamic Modeling and Simulation

Unmanned aerial vehicles (UAVs) have become a popular and useful tool for aerial mapping in a variety of domains. A limitation to even further adoption is the lack of scalability in many current methods, which require considerable manual work to incorporate new data. Inconsistent collection and processing of data can additionally prevent later consolidation and comparison of data across numerous dates and sites. An automated image processing workflow has been developed to allow end-users to plan and execute UAV flights with the end goal of publishing consistent and reliable data products. An example use case focusing on object detection for plant phenotyping shows the effectiveness of this workflow. Here, it is demonstrated that by utilizing this workflow, an end user is capable of producing consistent quality training images and annotations in a standardised way, allowing for the fast production and inference of new application models. As more data is adopted into the workflow, these models can be further improved by leveraging the additional data, which can reduce future training requirements and allow for progressively faster and more robust inference.

Summary The problem of non-random sample selectivity often occurs in practice in many fields. The classical estimators introduced by Heckman are the backbone of the standard statistical analysis of these models. However, these estimators are very sensitive to small deviations from the distributional assumptions which are often not satisfied in practice. We develop a general framework to study the robustness properties of estimators and tests in sample selection models. We derive the influence function and the change-of-variance function of Heckman's two-stage estimator, and we demonstrate the non-robustness of this estimator and its estimated variance to small deviations from the model assumed. We propose a procedure for robustifying the estimator, prove its asymptotic normality and give its asymptotic variance. Both cases with and without an exclusion restriction are covered. This allows us to construct a simple robust alternative to the sample selection bias test. We illustrate the use of our new methodology in an analysis of ambulatory expenditures and we compare the performance of the classical and robust methods in a Monte Carlo simulation study.

In many linear regression models, there are functional relationships among the covariates. The usual (partial) F tests for these models do not utilize these functional relationships, however, and so have certain undesirable properties. Improvements over these F tests are proposed in this article, and they are based on simultaneous confidence bands. For ease of illustration of the ideas, we focus on the polynomial regression model.