Abstract Randomized controlled trials (RCTs) are the gold standard for evaluating the causal effect of a treatment; however, they often have limited sample sizes and sometimes poor generalizability. On the other hand, non-randomized, observational data derived from large administrative databases have massive sample sizes and better generalizability, but are prone to unmeasured confounding bias. It is thus of considerable interest to reconcile effect estimates obtained from RCTs and observational studies investigating the same intervention, potentially harvesting the best from both realms. In this article, we theoretically characterize the potential efficiency gain from integrating observational data into the RCT-based analysis from a minimax perspective. For estimation, we derive the minimax rate of convergence for the mean-squared error and propose adaptive estimators that attain the optimal rate up to poly-log factors. For inference, we characterize the minimax rate for the length of confidence intervals and show that adaptation (to unknown confounding bias) is in general impossible. A curious phenomenon thus emerges: for estimation, the efficiency gain from data integration can be achieved without prior knowledge of the magnitude of the confounding bias; for inference, the same task becomes information theoretically impossible in general. We corroborate our theoretical findings using simulations and a real data example from the RCT DUPLICATE initiative.

Abstract Fixed effects estimation, with linear controls for stratum membership, is often used to estimate treatment effects when assignment propensities differ across strata. In the presence of heterogeneity in treatment effects across strata, this estimator does not target the average treatment effect, however. Indeed, the implied estimand can range anywhere from the lowest to the highest stratum-level average effect. To facilitate the interpretation of results using this approach, I establish that if stratum-level average effects are monotonic in the shares assigned to treatment, then the fixed effects estimand lies between the average treatment effect for the treated and the average treatment effect for the controls.

Abstract Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational datasets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive information might be tolerable to organizations that house confidential data. In these cases, organizations can employ data privacy techniques, which decrease disclosure risk, potentially at the expense of data utility. In this study, we explore disclosure limiting transformations of observational data, which can be combined with experimental data to estimate the sample and population average treatment effects. We consider leveraging observational data to improve generalizability of treatment effect estimates, when a randomized controlled trial (RCT) is not representative of the population of interest, and to increase precision of treatment effect estimates. Through simulation studies, we illustrate the trade-off between privacy and utility when employing different disclosure limiting transformations. We find that leveraging transformed observational data in treatment effect estimation can still improve estimation over only using data from an RCT.

Abstract In this article, we discuss both prospective and retrospective causal inferences, building on Neyman’s potential outcome framework. For prospective causal inference, we review criteria for confounders and surrogates to avoid the Yule–Simpson paradox and the surrogate paradox, respectively. For retrospective causal inference, we introduce the concepts of posterior causal effects given observed evidence to quantify the causes of effects. The posterior causal effects provide a unified framework for deducing both effects of causes in prospective causal inference and causes of effects in retrospective causal inference. We compare the medical diagnostic approaches based on Bayesian posterior probabilities and posterior causal effects for classification and attribution.

Abstract In causal inference, interference takes place when the intervention on one unit affects the outcome of other units. Most of the previous methods for estimating network causal effects assume that the covariate information is complete, which may lead to biased estimates when missingness exists. In this study, we consider the partial and direct interference setting. Specifically, the whole population can be divided into different clusters. Within each cluster, the outcome of each unit is dependent on the intervention received by other units, but not dependent on the confounders or outcomes of other units within the same cluster or of those in different clusters. We also assume that the confounders are subject to non-ignorable missingness, and a confounder is considered as missing if any component of it is missing. We propose three consistent estimators for the direct, indirect, total, and overall effect of the intervention on the outcome, and derive the asymptotic results accordingly. A comprehensive study is carried out as well to investigate the finite sample properties of the proposed estimators. We illustrate the proposed methods by analyzing the dataset collected from an acid rain program, which was launched to reduce air pollution in the United States by encouraging the scrubber’s installation on power plants, where the records of some operating characteristics of the power generating facilities are subject to missingness.

Abstract There is a long-standing debate in the statistical, epidemiological, and econometric fields as to whether nonparametric estimation that uses machine learning in model fitting confers any meaningful advantage over simpler, parametric approaches in finite sample estimation of causal effects. We address the question: when estimating the effect of a treatment on an outcome, how much does the choice of nonparametric vs parametric estimation matter? Instead of answering this question with simulations that reflect a few chosen data scenarios, we propose a novel approach to compare estimators across a large number of data-generating mechanisms drawn from nonparametric models with semi-informative priors. We apply this proposed approach and compare the performance of two nonparametric estimators (Bayesian adaptive regression tree and a targeted minimum loss-based estimator) to two parametric estimators (a logistic regression-based plug-in estimator and a propensity score estimator) in terms of estimating the average treatment effect across thousands of data-generating mechanisms. We summarize performance in terms of bias, confidence interval coverage, and mean squared error. We find that the two nonparametric estimators can substantially reduce bias as compared to the two parametric estimators in large-sample settings characterized by interactions and nonlinearities while compromising very little in terms of performance even in simple, small-sample settings.

Abstract We propose quantitative probing as a model-agnostic framework for validating causal models in the presence of quantitative domain knowledge. The method is constructed in analogy to the train/test split in correlation-based machine learning. It is consistent with the logic of scientific discovery and enhances current causal validation strategies. The effectiveness of the method is illustrated using Pearl’s sprinkler example, before a thorough simulation-based investigation is conducted. Limits of the technique are identified by studying exemplary failing scenarios, which are furthermore used to propose a list of topics for future research and improvements of the presented version of quantitative probing. A guide for practitioners is included to facilitate the incorporation of quantitative probing in causal modelling applications. The code for integrating quantitative probing into causal analysis, as well as the code for the presented simulation-based studies of the effectiveness of quantitative probing are provided in two separate open-source Python packages.

Abstract We consider likelihood score-based methods for causal discovery in structural causal models. In particular, we focus on Gaussian scoring and analyze the effect of model misspecification in terms of non-Gaussian error distribution. We present a surprising negative result for Gaussian likelihood scoring in combination with nonparametric regression methods.

Abstract Bayesian causal inference in randomized experiments usually imposes model-based structure on potential outcomes. Yet causal inferences from randomized experiments are especially credible because they depend on a known assignment process, not a probability model of potential outcomes. In this article, I derive a randomization-based procedure for Bayesian inference of causal effects in a finite population setting. I formally show that this procedure satisfies Bayesian analogues of unbiasedness and consistency under weak conditions on a prior distribution. Unlike existing model-based methods of Bayesian causal inference, my procedure supposes neither probability models that generate potential outcomes nor independent and identically distributed random sampling. Unlike existing randomization-based methods of Bayesian causal inference, my procedure does not suppose that potential outcomes are discrete and bounded. Consequently, researchers can reap the benefits of Bayesian inference without sacrificing the properties that make inferences from randomized experiments especially credible in the first place.

Abstract Understanding the pathways whereby an intervention has an effect on an outcome is a common scientific goal. A rich body of literature provides various decompositions of the total intervention effect into pathway-specific effects. Interventional direct and indirect effects provide one such decomposition. Existing estimators of these effects are based on parametric models with confidence interval estimation facilitated via the nonparametric bootstrap. We provide theory that allows for more flexible, possibly machine learning-based, estimation techniques to be considered. In particular, we establish weak convergence results that facilitate the construction of closed-form confidence intervals and hypothesis tests and prove multiple robustness properties of the proposed estimators. Simulations show that inference based on large-sample theory has adequate small-sample performance. Our work thus provides a means of leveraging modern statistical learning techniques in estimation of interventional mediation effects.