Prospective and retrospective causal inferences based on the potential outcome framework

The central thesis of this book is that causal inferences based on careful integrative multi-method research designs have advantages over similar inferences that rely on only one method. A necessary first step in making this argument is showing that many applications of qualitative and quantitative methods in the social sciences share an underlying notion of causation. If causation in these two methodological traditions is fundamentally not the same sort of relation, then there is good reason to worry that they cannot contribute to a joint causal project. Indeed, given such fundamentally divergent worldviews, it is unclear that the methods could even contribute to testing each others’ assumptions – the central theme of integrative multi-method research designs. I argue below that the potential-outcomes framework, in which causal effects are characterized as the difference between what actually happens in a given case and what would have happened had that case been assigned to a different treatment category, can serve as that notion. After introducing this concept, I argue that stereotypically qualitative and quantitative causal thinking can be seen as sharing an underlying conceptual structure. The potential-outcomes framework is used to explicate causal models and hypotheses throughout the book. Existing work on the potential-outcomes framework has produced clear insights into the challenges of causal inference. Most famously, authors have pointed to the fundamental problem of causal inference: because it is impossible to observe both what does happen to a case in reality and what would have happened to that case had the independent variable been assigned a different value, no direct empirical observation of causation at the case level is possible (Holland 1986). Prior research also provides detailed findings regarding the assumptions necessary to get around this problem and make causal inferences in experiments, natural experiments, and quantitative observational studies (e.g. Morgan and Winship 2007). The last segment of this chapter reviews those findings with respect to regression-type analysis of observational studies, paying special attention to the assumptions about control variables necessary for regression-based causal inference. In particular, this literature shows that scholars need independent evidence that the regression does not suffer from confounders, colliders, or post-treatment bias. While these assumptions are not sufficient for causal inference, they are necessary and provide key openings for integrative multi-method research design.

This paper reviews recent research on causal inference with large-scale assessments in education from a Bayesian perspective. I begin by adopting the potential outcomes model of Rubin (J Educ Psychol 66:688–701, 1974) as a framework for causal inference that I argue is appropriate with large-scale educational assessments. I then discuss the elements of Bayesian inference arguing that methods and models of causal inference can benefit from the Bayesian approach to quantifying uncertainty. Next I outline one method of causal inference that I believe is fruitful for addressing causal questions with large-scale educational assessments within the potential outcomes framework— namely, propensity score analysis. I then discuss the quantification of uncertainty in propensity score analysis through a Bayesian approach. Next, I discuss a series of necessary conditions for addressing causal questions with large-scale educational assessments. The paper closes with a discussion of the implications for the design of large-scale educational assessments when the goal is in asking causal questions and warranting causal claims.

Quantifying the robustness of causal inferences: Sensitivity analysis for pragmatic social science

Causal inference is a permanent challenge topic in statistics, data science, and many other scientific fields.In this paper, we give an overview of statistical methods for causal inference. There are two main frameworks of causal inference: the potential outcome model and the causal network model. The potential outcome framework is used to evaluate causal effects of a known treatment or exposure variable on a given response or outcome variable. We review several commonly-used approaches in this framework for causal effect evaluation.The causal network framework is used to depict causal relationships among variables and the data generation mechanism in complex systems.We review two main approaches for structural learning: the constraint-based method and the score-based method.In the recent years, the evaluation of causal effects and the structural learning of causal networks are combined together.At the first stage, the hybrid approach learns a Markov equivalent class of causal networks from observed data;then at the second stage, it evaluates the causal effect for each causal network in the class; it also obtains a set of causal effects.The current frameworks of causal inference still have various demerits and disadvantages.We discuss these challenges and possible solutions in modern big data studies.

Fig. S1. Phylogenetic tree generated by Bayesian inference analyses based on the individual CaM, rpb1, rpb2 and tef1 (A–D) for species in Fusarium fujikuroi species complex (FFSC). Bayesian posterior probability (PP ≥ 0.9) and IQ-TREE bootstrap support values (ML ≥ 70 %) presented above branches as follows: PP/ ML. PP < 0.9 and ML < 70 % of branches marked with *. Newly generated sequences are in blue. Ex-type and ex-epitype strains are indicated with T and ET, respectively. The tree is rooted to F. nirenbergiae (CBS 840.88). <br />Fig. S2. Phylogenetic tree generated by Bayesian inference analyses based on the individual CaM, rpb2 and tef1 (A–C) for species in Fusarium incarnatum-equiseti species complex (FIESC). Bayesian posterior probability (PP ≥ 0.9) and IQ-TREE bootstrap support values (ML ≥ 70 %) presented above branches as follows: PP/ML. PP < 0.9 and ML < 70 % of branches marked with *. Newly generated sequences are in blue. Ex-type, exepitype and ex-neotype strains are indicated with T, ET and NT respectively. The tree is rooted to F. nodosum (CBS 201.63). <br />Fig. S3. Phylogenetic tree generated by Bayesian inference analyses based on the individual rpb2, tef1 and tub2 (A–C) for species in Fusarium lateritium species complex (FLSC). Bayesian posterior probability (PP ≥ 0.9) and IQ-TREE bootstrap support values (ML ≥ 70 %) presented above branches as follows: PP/ML. PP < 0.9 and ML < 70 % of branches marked with *. Newly generated sequences are in blue. Ex-type, ex-epitype and ex-neotype strains are indicated with T, ET and NT respectively. The tree is rooted to F. acuminatum (LC5227). <br />Fig. S4. Phylogenetic tree generated by Bayesian inference analyses based on the individual CaM, rpb1, rpb2 and tef1 (A–D) for species in Fusarium oxysporum species complex (FOSC). Bayesian posterior probability (PP ≥ 0.9) and IQ-TREE bootstrap support values (ML ≥ 70 %) presented above branches as follows: PP/ML. PP < 0.9 and ML < 70 % of branches marked with *. Newly generated sequences are in blue. Ex-type and exepitype strains are indicated with T and ET, respectively. The tree is rooted to F. campestre (KG158). <br />Fig. S5. Phylogenetic tree generated by Bayesian inference analyses based on the individual rpb1, rpb2 and tef1 (A–C) for species in Fusarium sambucinum species complex (FSAMSC). Bayesian posterior probability (PP ≥ 0.9) and IQ-TREE bootstrap support values (ML ≥ 70 %) presented above branches as follows: PP/ ML. PP < 0.9 and ML < 70 % of branches marked with *. Newly generated sequences are in blue and ex-type and ex-epitype strains are indicated with T and ET, respectively. The tree is rooted to F. kotabaruense (Indo172). <br />Fig. S6. Phylogenetic tree generated by Bayesian inference analyses based on the individual rpb1, rpb2 and tef1 (A–C) for species in Fusarium tricinctum species complex (FTSC). Bayesian posterior probability (PP ≥ 0.9) and IQ-TREE bootstrap support values (ML ≥ 70 %) presented above branches as follows: PP/ ML. PP < 0.9 and ML < 70 % of branches marked with *. Newly generated sequences are in blue. Ex-type, ex-epitype and exneotype strains are indicated with T, ET and NT respectively. The tree is rooted to F. foetens (CBS 110286). <br />Fig. S7. Phylogenetic tree generated by Bayesian inference analyses based on the individual ITS, LSU, rpb2 and tef1 (A–D) for species in Neocosmospora. Bayesian posterior probability (PP ≥ 0.9) and IQ-TREE bootstrap support values (ML ≥ 70 %) presented above branches as follows: PP/ML. PP < 0.9 and ML < 70 % of branches marked with *. Newly generated sequences are in blue. Ex-type and ex-epitype strains are indicated with T and ET, respectively. The tree is rooted to Setofusarium setosum (CBS 635.92). <br />Table S1. Details of all fusarioid isolates obtained in this study. <br />Table S2. Fusarioid species isolated from citrus with branch canker in different orchards across different provinces.

ABSTRACTBackgroundSurvey data are essential in paediatric epidemiology, providing valuable insights into child health outcomes. The potential outcomes framework has advanced causal inference using observational data. However, traditional design‐based adjustments, especially sample weights, are often overlooked. This omission limits the ability to generalise findings to the broader population.ObjectiveThis study demonstrates three approaches for estimating the population average treatment effect (PATE) in a practical example, examining the impact of household second‐hand smoke (SHS) exposure on blood pressure in school‐aged children.MethodsUsing data from the National Health and Nutrition Examination Survey (NHANES) 2017–2020, we assessed the effect of household SHS exposure, a non‐randomised treatment, on blood pressure in school‐aged children. We applied estimators based on Inverse Probability of Treatment Weighting (IPTW), G‐computation, Targeted Maximum Likelihood Estimation (TMLE), and regression adjustment. Models without adjustments were run for comparison. We examined point estimates and the efficiency of the estimates obtained from these methods.ResultsThe largest differences were observed between the unadjusted regression models and the fully adjusted methods (IPTW, G‐computation, and TMLE), which account for both confounding and survey weights. While the inclusion of the sample weights leads to wider confidence intervals for all methods, G‐computation and TMLE showed comparatively narrower confidence intervals. Confidence intervals for the models not adjusted for sample weights were likely underestimated.ConclusionsThis study highlights the important role of sample weights in causal inference. Generalisability of the average treatment effect as estimated on data sampled using common survey designs to a defined population requires the use of sample weights. The estimators described provide a framework for incorporating sample weights, and their use in health research is recommended.

The review by Ben-Shlomo et al.1 highlights how life course epidemiology is evolving and adapting to accommodate increasing access to data on novel dimensions and over extended periods. This enriched framework raises ever greater methodological challenges, leaving statisticians like us daunted by the task of translating life course enquiries into suitable analyses of the data at hand. Take for example Figure 4 of Ben-Shlomo et al..1 This is very useful for gaining a ‘big picture’ understanding of a complex area such as ageing, and for establishing which processes may benefit from a more detailed investigation. However, the leap from such a diagram to a specific data analysis should not be (and is not typically) made without greater thought. We will argue in this commentary that some recent developments from the field of modern causal inference may be helpful in this regard. First, in order to state unambiguously the question (or questions) of interest, the potential outcomes framework, a cornerstone of modern causal inference thinking, is invaluable. Then, the conceptual framework should be refined to a causal directed acyclic graph (DAG) relevant to the question, and the causal DAG should be formally interrogated to see if the question can be addressed, and if so how. Indeed, depending on the question, the causal DAG and the data available, we may find that standard statistical methods traditionally used in epidemiology are sufficient; in other settings we may find that more novel techniques are needed. We will discuss each of these points next, mentioning also the issues of missing data and measurement error, as well as highlighting concerns about the difference between the processes which are the focus of investigations and their manifestations in observed data.

The P-value is the significance probability of obtaining a value of the test statistic that is as extreme, in relation to the null hypothesis, as that observed. Medical researchers may, in some situations, disagree on its appropriate use or on its interpretation as a summary measure of consistency with the null hypothesis in a particular data set. More informative statistical measures such as the likelihood ratio and the Bayesian posterior probability have been suggested for drawing inferences from clinical trials and epidemiologic studies. Causal inference is not statistical in nature; rather it strives to provide scientific explanations or criticisms of proposed explanations that would describe the observed data pattern. In this context, it is important to remember that a finding may not be medically important, or a causal hypothesis may even not be true even if a study shows a significant P-value.

Many empirical studies have revealed considerable differences between nonparametric bootstrapping and Bayesian posterior probabilities in terms of the support values for branches, despite claimed predictions about their approximate equivalence. We investigated this problem by simulating data, which were then analyzed by maximum likelihood bootstrapping and Bayesian phylogenetic analysis using identical models and reoptimization of parameter values. We show that Bayesian posterior probabilities are significantly higher than corresponding nonparametric bootstrap frequencies for true clades, but also that erroneous conclusions will be made more often. These errors are strongly accentuated when the models used for analyses are underparameterized. When data are analyzed under the correct model, nonparametric bootstrapping is conservative. Bayesian posterior probabilities are also conservative in this respect, but less so.

Counterfactuals, graphical causal models and potential outcomes: Response to Lindquist and Sobel

Fig. S1. Phylogram obtained from ML analyses of the partial BT and EF gene sequences of the O. clavatum species complex. Sequences obtained in this study are printed in bold type. Maximum-likelihood bootstrap support values (1 000 replicates) above 70 % are indicated at the nodes. Bayesian inference posterior probabilities (above 0.9) are indicated by bold lines at the relevant branches. T = ex-type cultures. Fig. S2. Phylogram obtained from ML analyses of the ITS region and the partial BT gene of the O. ips species complex. Sequences obtained in this study are printed in bold type. Maximum-likelihood bootstrap support values (1 000 replicates) above 70 % are indicated at the nodes. The Bayesian inference posterior probabilities (above 0.9) are indicated by bold lines at the relevant branches. T = ex-type cultures. Fig. S3. Phylogram obtained from ML analyses of the ITS region, and the partial BT and CAL gene sequences of the Sporothrix gossypina species complex. Sequences obtained in this study are printed in bold type. Maximum-likelihood bootstrap support values (1 000 replicates) above 70 % are indicated at the nodes. Bayesian inference posterior probabilities (above 0.9) are indicated by bold lines at the relevant branches. T = ex-type cultures. Fig. S4. Phylogram obtained from ML analyses of the ITS region, and the partial BT and EF gene sequences of Endoconidiophra. Sequences obtained in this study are printed in bold type. Maximum-likelihood bootstrap support values (1 000 replicates) above 70 % are indicated at the nodes. Bayesian inference posterior probabilities (above 0.9) are indicated by bold lines at the relevant branches. T = ex-type cultures.

Evolutionary trees sit at the core of all realistic models describing a set of related sequences, including alignment, homology search, ancestral protein reconstruction and 2D/3D structural change. It is important to assess the stochastic error when estimating a tree, including models using the most realistic likelihood-based optimizations, yet computation times may be many days or weeks. If so, the bootstrap is computationally prohibitive. Here we show that the extremely fast "resampling of estimated log likelihoods" or RELL method behaves well under more general circumstances than previously examined. RELL approximates the bootstrap (BP) proportions of trees better that some bootstrap methods that rely on fast heuristics to search the tree space. The BIC approximation of the Bayesian posterior probability (BPP) of trees is made more accurate by including an additional term related to the determinant of the information matrix (which may also be obtained as a product of gradient or score vectors). Such estimates are shown to be very close to MCMC chain values. Our analysis of mammalian mitochondrial amino acid sequences suggest that when model breakdown occurs, as it typically does for sequences separated by more than a few million years, the BPP values are far too peaked and the real fluctuations in the likelihood of the data are many times larger than expected. Accordingly, several ways to incorporate the bootstrap and other types of direct resampling with MCMC procedures are outlined. Genes evolve by a process which involves some sites following a tree close to, but not identical with, the species tree. It is seen that under such a likelihood model BP (bootstrap proportions) and BPP estimates may still be reasonable estimates of the species tree. Since many of the methods studied are very fast computationally, there is no reason to ignore stochastic error even with the slowest ML or likelihood based methods.

We perform inference on the effect of a treatment on survival times in studies where the treatment assignment is not randomized and the assignment time is not known in advance. We estimate survival functions on a treated and a control group which are made comparable through matching on observed covariates. The inference is performed by conditioning on waiting time to treatment, that is time between the entrance in the study and treatment. This can be done only when sufficient data is available. In other cases, averaging over waiting times is a possibility, although the classical interpretation of the estimated survival functions is lost unless hazards are not functions of the waiting times. To show unbiasedness and to obtain an estimator of the variance, we build on the potential outcome framework, which was introduced by J. Neyman in the context of randomized experiments, and adapted to observational studies by D. B. Rubin. Our approach does not make parametric or distributional assumptions. In particular, we do not assume proportionality of the hazards compared. Small sample performance of the estimator and a derived test of no treatment effect are studied in a Monte Carlo study. We perform inference on the effect of a treatment on survival times in studies where the treatment assignment is not randomized and the assignment time is not known in advance. We estimate survival functions on a treated and a control group which are made comparable through matching on observed covariates. The inferenceis performed by conditioning on waiting time to treatment, that is time between the entrance in the study and treatment. This can be done only when sufficient data is available. In other cases, averaging over waiting times is a possibility, although the classical interpretation of the estimated survival functions is lost unless hazards are not functions of the waiting times. To show unbiasedness and to obtain an estimator of the variance, we build on the potential outcome framework, which was introduced by J. Neyman in the context of randomized experiments, and adapted to observational studies by D. B. Rubin. Our approach does not make parametric or distributional assumptions. In particular, we do not assume proportionality of the hazards compared. Small sample performance of the estimator and a derived test of no treatment effect are studied in a Monte Carlo study.

BackgroundLogistic regression is often used for mediation analysis with a dichotomous outcome. However, previous studies showed that the indirect effect and proportion mediated are often affected by a change of scales in logistic regression models. To circumvent this, standardization has been proposed. The aim of this study was to show the relative performance of the unstandardized and standardized estimates of the indirect effect and proportion mediated based on multiple regression, structural equation modeling, and the potential outcomes framework for mediation models with a dichotomous outcome.MethodsWe compared the performance of the effect estimates yielded by the three methods using a simulation study and two real-life data examples from an observational cohort study (n = 360).ResultsLowest bias and highest efficiency were observed for the estimates from the potential outcomes framework and for the crude indirect effect ab and the proportion mediated ab/(ab + c’) based on multiple regression and SEM.ConclusionsWe advise the use of either the potential outcomes framework estimates or the ab estimate of the indirect effect and the ab/(ab + c’) estimate of the proportion mediated based on multiple regression and SEM when mediation analysis is based on logistic regression. Standardization of the coefficients prior to estimating the indirect effect and the proportion mediated may not increase the performance of these estimates.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2022 SIAM International Conference on Data Mining (SDM)Learning Infomax and Domain-Independent Representations for Causal Effect Inference with Real-World DataZhixuan Chu, Stephen L. Rathbun, and Sheng LiZhixuan Chu, Stephen L. Rathbun, and Sheng Lipp.433 - 441Chapter DOI:https://doi.org/10.1137/1.9781611977172.49PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract The foremost challenge to causal inference with real-world data is to handle the imbalance in the covariates with respect to different treatment options, caused by treatment selection bias. To address this issue, recent literature has explored domain-invariant representation learning based on different domain divergence metrics (e.g., Wasserstein distance, maximum mean discrepancy, position-dependent metric, and domain overlap). In this paper, we reveal the weaknesses of these strategies, i.e., they lead to the loss of predictive information when enforcing the domain invariance; and the treatment effect estimation performance is unstable, which heavily relies on the characteristics of the domain distributions and the choice of domain divergence metrics. Motivated by information theory, we propose to learn the Infomax and Domain-Independent Representations to solve the above puzzles. Our method utilizes the mutual information between the global feature representations and individual feature representations, and the mutual information between feature representations and treatment assignment predictions, in order to maximally capture the common predictive information for both treatment and control groups. Moreover, our method filters out the influence of instrumental and irrelevant variables, and thus it effectively increases the predictive ability of potential outcomes. Experimental results on both the synthetic and real-world datasets show that our method achieves state-of-the-art performance on causal effect inference. Moreover, our method exhibits reliable prediction performances when facing data with different characteristics of data distributions, complicated variable types, and severe covariate imbalance. Previous chapter Next chapter RelatedDetails Published:2022eISBN:978-1-61197-717-2 https://doi.org/10.1137/1.9781611977172Book Series Name:ProceedingsBook Code:PRDT22Book Pages:1-737Key words:causal inference, selection bias, mutual information, real-world data