Streaming feature-based causal structure learning algorithm with symmetrical uncertainty

Causal knowledge is vital for effective reasoning in science, as causal relations, unlike correlations, allow one to reason about the outcomes of interventions. Algorithms that can discover causal relations from observational data are based on the assumption that all variables have been jointly measured in a single dataset. In many cases this assumption fails. Previous approaches to overcoming this shortcoming devised algorithms that returned all joint causal structures consistent with the conditional independence information contained in each individual dataset. But, as conditional independence tests only determine causal structure up to Markov equivalence, the number of consistent joint structures returned by these approaches can be quite large. The last decade has seen the development of elegant algorithms for discovering causal relations beyond conditional independence, which can distinguish among Markov equivalent structures. In this work we adapt and extend these so-called bivariate causal discovery algorithms to the problem of learning consistent causal structures from multiple datasets with overlapping variables belonging to the same generating process, providing a sound and complete algorithm that outperforms previous approaches on synthetic and real data.

Based on benchmarking various methods and analyzing multiple real scRNA-seq datasets, a computational platform/workflow and a set of tips for best practices are developed for analyzing causal interactions or relationships in single cells.

Application of high-throughput data from microarray experiments is a promising way of finding regulatory relationships between genes. Recently, a causal model approach was presented for learning gene regulatory networks from microarray data. The paper investigates and extends this model by identifying the way in which the causal structure can be best exploited computationally to make inference faster and closer to the underlying model. We examine the influence of various hypotheses about the sub-model space on the discovery of causal models. We develop scoring algorithms to learn V-structure, Y- structure and the Markov blanket of every node, by the efficient application of conditional independence (CI) tests of increasing orders on the data. Data from artificial networks as well as real-world S.cerevisiae cellcycle measurements (Spellman et al) were used to evaluate the confidence in our scoring algorithms. The Markov blanket algorithm demonstrated better structural correctness, computational efficiency along with accuracy improvement compared to V-structure and Y-structure learning algorithms.

It is a common saying that testing for conditional independence, that is, testing whether whether two random vectors $X$ and $Y$ are independent, given $Z$, is a hard statistical problem if $Z$ is a continuous random variable (or vector). In this paper, we prove that conditional independence is indeed a particularly difficult hypothesis to test for. Valid statistical tests are required to have a size that is smaller than a pre-defined significance level, and different tests usually have power against a different class of alternatives. We prove that a valid test for conditional independence does not have power against any alternative. Given the nonexistence of a uniformly valid conditional independence test, we argue that tests must be designed so their suitability for a particular problem may be judged easily. To address this need, we propose in the case where $X$ and $Y$ are univariate to nonlinearly regress $X$ on $Z$, and $Y$ on $Z$ and then compute a test statistic based on the sample covariance between the residuals, which we call the generalised covariance measure (GCM). We prove that validity of this form of test relies almost entirely on the weak requirement that the regression procedures are able to estimate the conditional means $X$ given $Z$, and $Y$ given $Z$, at a slow rate. We extend the methodology to handle settings where $X$ and $Y$ may be multivariate or even high dimensional. While our general procedure can be tailored to the setting at hand by combining it with any regression technique, we develop the theoretical guarantees for kernel ridge regression. A simulation study shows that the test based on GCM is competitive with state of the art conditional independence tests. Code is available as the R package $\mathtt{GeneralisedCovarianceMeasure}$ on CRAN.

Network psychometrics uses graphical models to assess the network structure of psychological variables. An important task in their analysis is determining which variables are unrelated in the network, i.e., are independent given the rest of the network variables. This conditional independence structure is a gateway to understanding the causal structure underlying psychological processes. Thus, it is crucial to have an appropriate method for evaluating conditional independence and dependence hypotheses. Bayesian approaches to testing such hypotheses allow researchers to differentiate between absence of evidence and evidence of absence of connections (edges) between pairs of variables in a network. Three Bayesian approaches to assessing conditional independence have been proposed in the network psychometrics literature. We believe that their theoretical foundations are not widely known, and therefore we provide a conceptual review of the proposed methods and highlight their strengths and limitations through a simulation study. We also illustrate the methods using an empirical example with data on Dark Triad Personality. Finally, we provide recommendations on how to choose the optimal method and discuss the current gaps in the literature on this important topic.

OFS-Density: A novel online streaming feature selection method

Causality-informed machine learning has been proposed as an avenue for achieving many of the goals of modern machine learning, from ensuring generalization under domain shifts to attaining fairness, robustness, and interpretability. A key component of causal machine learning is the inference of causal structures from observational data; in practice, this data may be incompletely observed. Prior work has demonstrated that adversarial perturbations of completely observed training data may be used to force the learning of inaccurate structural causal models (SCMs). However, when the data can be audited for correctness (e.g., it is cryptographically signed by its source), this adversarial mechanism is invalidated. This work introduces a novel attack methodology wherein the adversary deceptively omits a portion of the true training data to bias the learned causal structures in a desired manner (under strong signed sample input validation, this behavior seems to be the only strategy available to the adversary). Under this model, theoretically sound attack mechanisms are derived for the case of arbitrary SCMs, and a sample-efficient learning-based heuristic is given. Experimental validation of these approaches on real and synthetic datasets, across a range of SCMs from the family of additive noise models (linear Gaussian, linear non-Gaussian, and non-linear Gaussian), demonstrates the effectiveness of adversarial missingness attacks at deceiving popular causal structure learning algorithms.

Conditional independence (CI) testing is an important problem, especially in causal discovery. Most testing methods assume that all variables are fully observable and then test the CI among the observed data. Such an assumption is often untenable beyond applications dealing with, e.g., psychological analysis about the mental health status and medical diagnosing (researchers need to consider the existence of latent variables in these scenarios); and typically adopted latent CI test schemes mainly suffer from robust or efficient issues. Accordingly, this article investigates the problem of testing CI between latent variables. To this end, we offer an auxiliary regression-based CI (AReCI) test by taking the measured variable as the surrogate variable of the latent variables to conduct the regression over the latent variables under the linear causal models, in which each latent variable has some certain measured variables. Specifically, given a pair of latent variables LX and LY , and a corresponding latent variable set LO , [Formula: see text] holds if and only if [Formula: see text] and [Formula: see text] are statistically independent, where A' and A'' are the two disjoint subset of the measured variable for the corresponding latent variables, A'{LO} ∩A''{LO} = ∅ , and ω1 is a parameter vector characterized from the cross covariance between A{LX} and A'{LO} , and ω2 is a parameter vector characterized from the cross covariance between A{LY} and A''{LO} . We theoretically show that the AReCI test is capable of addressing both Gaussian and non-Gaussian data. In addition, we find that the well-known partial correlation test can be seen as a special case of the AReCI test. Finally, we devise a causal discovery method by using the AReCI test as the CI test. The experimental results on synthetic and real-world data illustrate the effectiveness of our method.

Current estimates of the contribution of education to economic growth have been questioned because they ignore the interaction of education with ability. Whether the neglect of ability differences in the analyses of the income-education relationship results in estimates that are too high was considered in an earlier paper by one of the authors (Griliches 1970), and a negative answer was conjectured. In this paper, we pursue this question a bit further, using a new and larger body of data. Unfortunately, a definitive answer to this question is hampered both by the vagueness and elasticity of "education" and "ability" as analytical concepts and by the lack of data on early (preschooling) intelligence. The data examined in this paper are based on a 1964 sample of U.S. military veterans. The variables measured include scores on a mental ability test, indicators of parental status, region of residence while growing up, school years completed before service, and school years completed during or after service. These have allowed us to inquire into the separate effects of parental background, intelligence, and schooling. The basic problem and analytical framework can be set out very simply. Let income be a linear function of education and ability, or, Y a + P1E + P(2G + u, where Y is income, E is education, G is ability,

Learning Bayesian networks from data: An information-theory based approach

Deep Neural Networks (DNNs) are often examined at the level of their response to input, such as analyzing the mutual information between nodes and data sets. Yet DNNs can also be examined at the level of causation, exploring “what does what” within the layers of the network itself. Historically, analyzing the causal structure of DNNs has received less attention than understanding their responses to input. Yet definitionally, generalizability must be a function of a DNN’s causal structure as it reflects how the DNN responds to unseen or even not-yet-defined future inputs. Here, we introduce a suite of metrics based on information theory to quantify and track changes in the causal structure of DNNs during training. Specifically, we introduce the effective information (EI) of a feedforward DNN, which is the mutual information between layer input and output following a maximum-entropy perturbation. The EI can be used to assess the degree of causal influence nodes and edges have over their downstream targets in each layer. We show that the EI can be further decomposed in order to examine the sensitivity of a layer (measured by how well edges transmit perturbations) and the degeneracy of a layer (measured by how edge overlap interferes with transmission), along with estimates of the amount of integrated information of a layer. Together, these properties define where each layer lies in the “causal plane”, which can be used to visualize how layer connectivity becomes more sensitive or degenerate over time, and how integration changes during training, revealing how the layer-by-layer causal structure differentiates. These results may help in understanding the generalization capabilities of DNNs and provide foundational tools for making DNNs both more generalizable and more explainable.

Background/Aim: Identifying confounders based on causal directed acyclic graphs (DAGs) is usually based solely on expert knowledge, although epidemiological studies provide observational data. We applied data-based structure learning algorithms for the specific purpose of identifying confounders, investigating the research question whether particulate matter with an aerodynamic diameter ≤2.5 µg/m3 (PM2.5) affects progression of atherosclerosis, measured as yearly change in carotid intima media thickness (ΔcIMT). Methods: We used 5-year follow-up data of the population-based Heinz Nixdorf Recall Study. Due to computational limitations, potentially relevant variables were restricted to continuously measured data and hence included age, BMI, smoking, blood pressure, HbA1c, LDL/HDL, education, income, neighborhood unemployment (nSES) rate, physical activity and alcohol consumption. Prior knowledge was based on causal chain assumptions regarding major causes of ischaemic heart disease1, and translated to prohibited arcs, i.e. given conditions (e.g. age) are unaffected by lifestyle variables (e.g. smoking) and lifestyle variables are unaffected by subclinical medical determinants (e.g. laboratory measures). We applied a revised structure learning algorithm to learn a causal DAG, including the score-based hill-climbing algorithm, a bootstrap-check to validate the conditional independence structure between variables, and a clustering of minimal sufficient adjustment set (MSAS)-equivalent subsets within the Markov equivalence-class. Results: Based on 2279 participants, the revised structure learning resulted in one MSAS-equivalent cluster, including one DAG of 59 unambiguous arcs. The only biasing path from PM2.5 to ΔcIMT emanated from nSES. Suggested adjustment consisted of four confounder-equivalent MSAS ({nSES}; {age, physical activity, blood pressure}; {age, alcohol consumption, BMI, physical activity, smoking}; {age, BMI, education, income, physical activity, smoking}), yielding an identical exposure effect estimate of 0.002 mm (95%-CI: -0.002, 0.007) increase in cIMT per year per 5 µg/m3 increase in PM2.5. Conclusions: Data-based structure learning returned one unambiguous DAG widely matching general believes. Identical exposure effect estimates of confounder-equivalent MSAS underline a stable (causal) structure.

A consistent characteristic function-based test for conditional independence

Exploring and detecting the causal relations among variables have shown huge practical values in recent years, with numerous opportunities for scientific discovery, and have been commonly seen as the core of data science. Among all possible causal discovery methods, causal discovery based on a constraint approach could recover the causal structures from passive observational data in general cases, and had shown extensive prospects in numerous real world applications. However, when the graph was sufficiently large, it did not work well. To alleviate this problem, an improved causal structure learning algorithm named brain storm optimization (BSO), is presented in this paper, combining K2 with brain storm optimization (K2-BSO). Here BSO is used to search optimal topological order of nodes instead of graph space. This paper assumes that dataset is generated by conforming to a causal diagram in which each variable is generated from its parent based on a causal mechanism. We designed an elaborate distance function for clustering step in BSO according to the mechanism of K2. The graph space therefore was reduced to a smaller topological order space and the order space can be further reduced by an efficient clustering method. The experimental results on various real-world datasets showed our methods outperformed the traditional search and score methods and the state-of-the-art genetic algorithm-based methods.

. A new metric, called variation of conditional mean (VCM), is proposed to measure the dependence of conditional mean of a response variable on a predictor variable. The VCM has several appealing merits. It equals zero if and only if the conditional mean of the response is independent of the predictor; it can be used for both real vector-valued variables and functional data. An estimator of the VCM is given through kernel smoothing, and a test for the conditional mean independence based on the estimated VCM is constructed. The limit distributions of the test statistic under the null hypothesis and alternative hypothesis are deduced, respectively. We further use VCM as a marginal utility to do high-dimensional feature screening to screen out variables that do not contribute to the conditional mean of the response given the predictors and prove the validity of the sure screening property. Furthermore, we find the cross variation of conditional mean (CVCM), a variant of the VCM, has a faster convergence rate than the VCM under conditional mean independence. Numerical comparison shows that the VCM and CVCM performs well in both conditional independence testing and feature screening. We also illustrate their applications to real data sets.