Integrating Overlapping Datasets Using Bivariate Causal Discovery

Based on benchmarking various methods and analyzing multiple real scRNA-seq datasets, a computational platform/workflow and a set of tips for best practices are developed for analyzing causal interactions or relationships in single cells.

Streaming feature-based causal structure learning algorithm with symmetrical uncertainty

It is a common saying that testing for conditional independence, that is, testing whether whether two random vectors $X$ and $Y$ are independent, given $Z$, is a hard statistical problem if $Z$ is a continuous random variable (or vector). In this paper, we prove that conditional independence is indeed a particularly difficult hypothesis to test for. Valid statistical tests are required to have a size that is smaller than a pre-defined significance level, and different tests usually have power against a different class of alternatives. We prove that a valid test for conditional independence does not have power against any alternative. Given the nonexistence of a uniformly valid conditional independence test, we argue that tests must be designed so their suitability for a particular problem may be judged easily. To address this need, we propose in the case where $X$ and $Y$ are univariate to nonlinearly regress $X$ on $Z$, and $Y$ on $Z$ and then compute a test statistic based on the sample covariance between the residuals, which we call the generalised covariance measure (GCM). We prove that validity of this form of test relies almost entirely on the weak requirement that the regression procedures are able to estimate the conditional means $X$ given $Z$, and $Y$ given $Z$, at a slow rate. We extend the methodology to handle settings where $X$ and $Y$ may be multivariate or even high dimensional. While our general procedure can be tailored to the setting at hand by combining it with any regression technique, we develop the theoretical guarantees for kernel ridge regression. A simulation study shows that the test based on GCM is competitive with state of the art conditional independence tests. Code is available as the R package $\mathtt{GeneralisedCovarianceMeasure}$ on CRAN.

. A new metric, called variation of conditional mean (VCM), is proposed to measure the dependence of conditional mean of a response variable on a predictor variable. The VCM has several appealing merits. It equals zero if and only if the conditional mean of the response is independent of the predictor; it can be used for both real vector-valued variables and functional data. An estimator of the VCM is given through kernel smoothing, and a test for the conditional mean independence based on the estimated VCM is constructed. The limit distributions of the test statistic under the null hypothesis and alternative hypothesis are deduced, respectively. We further use VCM as a marginal utility to do high-dimensional feature screening to screen out variables that do not contribute to the conditional mean of the response given the predictors and prove the validity of the sure screening property. Furthermore, we find the cross variation of conditional mean (CVCM), a variant of the VCM, has a faster convergence rate than the VCM under conditional mean independence. Numerical comparison shows that the VCM and CVCM performs well in both conditional independence testing and feature screening. We also illustrate their applications to real data sets.

Network psychometrics uses graphical models to assess the network structure of psychological variables. An important task in their analysis is determining which variables are unrelated in the network, i.e., are independent given the rest of the network variables. This conditional independence structure is a gateway to understanding the causal structure underlying psychological processes. Thus, it is crucial to have an appropriate method for evaluating conditional independence and dependence hypotheses. Bayesian approaches to testing such hypotheses allow researchers to differentiate between absence of evidence and evidence of absence of connections (edges) between pairs of variables in a network. Three Bayesian approaches to assessing conditional independence have been proposed in the network psychometrics literature. We believe that their theoretical foundations are not widely known, and therefore we provide a conceptual review of the proposed methods and highlight their strengths and limitations through a simulation study. We also illustrate the methods using an empirical example with data on Dark Triad Personality. Finally, we provide recommendations on how to choose the optimal method and discuss the current gaps in the literature on this important topic.

Application of high-throughput data from microarray experiments is a promising way of finding regulatory relationships between genes. Recently, a causal model approach was presented for learning gene regulatory networks from microarray data. The paper investigates and extends this model by identifying the way in which the causal structure can be best exploited computationally to make inference faster and closer to the underlying model. We examine the influence of various hypotheses about the sub-model space on the discovery of causal models. We develop scoring algorithms to learn V-structure, Y- structure and the Markov blanket of every node, by the efficient application of conditional independence (CI) tests of increasing orders on the data. Data from artificial networks as well as real-world S.cerevisiae cellcycle measurements (Spellman et al) were used to evaluate the confidence in our scoring algorithms. The Markov blanket algorithm demonstrated better structural correctness, computational efficiency along with accuracy improvement compared to V-structure and Y-structure learning algorithms.

Conditional independence (CI) testing is an important problem, especially in causal discovery. Most testing methods assume that all variables are fully observable and then test the CI among the observed data. Such an assumption is often untenable beyond applications dealing with, e.g., psychological analysis about the mental health status and medical diagnosing (researchers need to consider the existence of latent variables in these scenarios); and typically adopted latent CI test schemes mainly suffer from robust or efficient issues. Accordingly, this article investigates the problem of testing CI between latent variables. To this end, we offer an auxiliary regression-based CI (AReCI) test by taking the measured variable as the surrogate variable of the latent variables to conduct the regression over the latent variables under the linear causal models, in which each latent variable has some certain measured variables. Specifically, given a pair of latent variables LX and LY , and a corresponding latent variable set LO , [Formula: see text] holds if and only if [Formula: see text] and [Formula: see text] are statistically independent, where A' and A'' are the two disjoint subset of the measured variable for the corresponding latent variables, A'{LO} ∩A''{LO} = ∅ , and ω1 is a parameter vector characterized from the cross covariance between A{LX} and A'{LO} , and ω2 is a parameter vector characterized from the cross covariance between A{LY} and A''{LO} . We theoretically show that the AReCI test is capable of addressing both Gaussian and non-Gaussian data. In addition, we find that the well-known partial correlation test can be seen as a special case of the AReCI test. Finally, we devise a causal discovery method by using the AReCI test as the CI test. The experimental results on synthetic and real-world data illustrate the effectiveness of our method.

A consistent characteristic function-based test for conditional independence

We investigate the relationship between conditional independence (CI) x ⊥ y|Z and the independence of two residuals x – E(x|Z) ⊥ –E(y|Z), where x and y are two random variables, and Z is a set of random variables. We show that if x, y and Z are generated by following linear structural equation model and all external influences follow Gaussian distributions, then x ⊥ y|Z if and only if x – E(x|Z) ⊥ y – E(y|Z). That is, the test of x ⊥ y|Z can be relaxed to a simpler unconditional independence test of x – E(x|Z) ⊥ y – E(y|Z). Furthermore, if all these external influences follow non-Gaussian distributions and the model satisfies structural faithfulness condition, then we have x ⊥ y|Z ⇔ x – E(x|Z) ⊥ y – E(y|Z). We apply the results above to the causal discovery problem, where the causal directions are generally determined by a set of V-structures and their consistent propagations, so CI test-based methods can return a set of Markov equivalence classes. We show that in linear non-Gaussian context, x – E(x|Z) ⊥ y – E(y|Z) ⇒ x – E(x|Z) ⊥ z or y – E(y|Z ⊥ z (∀z ∈ Z) if Z is a minimal d-separator, which implies z causes x (or y) if z directly connects to x (or y). Therefore, we conclude that CIs have useful information for distinguishing Markov equivalence classes. In summary, compared with the existing discretization-based and kernel-based CI testing methods, the proposed method provides a simpler way to measure CI, which needs only one unconditional independence test and two regression operations. When being applied to causal discovery, it can find more causal relationships, which is experimentally validated.

Recently, regression based conditional independence (CI) tests have been employed to solve the problem of causal discovery. These methods provide an alternative way to test for CI by transforming CI to independence between residuals. Generally, it is nontrivial to check for independence when these residuals are linearly uncorrelated. With the ability to represent high-order moments, kernel-based methods are usually used to achieve this goal, but at a cost of considerable time. In this paper, we investigate the independence between two linear combinations under linear non-Gaussian structural equation model (SEM). We show that generally the 1-st to 4-th moments of the two linear combinations contain enough information to infer whether or not they are independent. The proposed method provides a simpler but more effective way to measure CIs, with only calculating the 1-st to 4-th moments of the input variables. When applied to causal discovery, the proposed method outperforms kernel-based methods in terms of both speed and accuracy. which is validated by extensive experiments.

In the genotyped-proband design, a proband is selected based on an observed phenotype, the genotype of the proband is observed, and then the phenotypes of all first-degree relatives are obtained. The genotypes of these first-degree relatives are not observed. Gail et al. [(1999) Genet Epidemiol] discuss likelihood analysis of this design under the assumption that the phenotypes are conditionally independent of one another given the observed and unobserved genotypes. Li and Thompson [(1997) Biometrics 53:282–293] give an example where this assumption is suspect, thus suggesting that it is important to develop tests for conditional independence. In this paper, we develop a score test for the conditional independence assumption in models that might include covariates or observation of genotypes for some of the first degree relatives. The problem can be cast more generally as one of score testing in the presence of missing covariates. A standard analysis would require specifying a distribution for the covariates, which is not convenient and could lead to a lack of model-robustness. We show that by considering a natural conditional likelihood, and basing the score test on it, a simple analysis results. The methods are applied to a study of the penetrance for breast cancer of BRCA1 and BRCA2 mutations among Ashkenazi Jews. Genet. Epidemiol. 18:293–306, 2000. © 2000 Wiley-Liss, Inc.

The type of constraint-based methods is one of the most important approaches to learn Bayesian network (BN) structures from observational data with conditional independence (CI) tests. In this paper, we find that existing constraint-based methods often perform many redundant CI tests, which significantly reduces the learning efficiency of those algorithms. To tackle this issue, we propose a novel framework to accelerate BN structure learning by reducing redundant CI tests without sacrificing accuracy. Specifically, we first design a CI test cache table to store CI tests. If a CI test has been computed before, the result of the CI test is obtained from the table instead of computing the CI test again. If not, the CI test is computed and stored in the table. Then based on the table, we propose two CI test cache table based PC (CTPC) learning frameworks for reducing redundant CI tests for BN structure learning. Finally, we instantiate the proposed frameworks with existing well-established local and global BN structure learning algorithms. Using twelve benchmark BNs, the extensive experiments have demonstrated that the proposed frameworks can significantly accelerate existing BN structure learning algorithms without sacrificing accuracy.

MotivationDeciphering gene interaction networks (GINs) from time-course gene expression (TCGx) data is highly valuable to understand gene behaviors (e.g., activation, inhibition, time-lagged causality) at the system level. Existing methods usually use a global or local proximity measure to infer GINs from a single dataset. As the noise contained in a single data set is hardly self-resolved, the results are sometimes not reliable. Also, these proximity measurements cannot handle the co-existence of the various in vivo positive, negative and time-lagged gene interactions.Methods and resultsWe propose to infer reliable GINs from multiple TCGx datasets using a novel conserved subsequential pattern of gene expression. A subsequential pattern is a maximal subset of genes sharing positive, negative or time-lagged correlations of one expression template on their own subsets of time points. Based on these patterns, a GIN can be built from each of the datasets. It is assumed that reliable gene interactions would be detected repeatedly. We thus use conserved gene pairs from the individual GINs of the multiple TCGx datasets to construct a reliable GIN for a species. We apply our method on six TCGx datasets related to yeast cell cycle, and validate the reliable GINs using protein interaction networks, biopathways and transcription factor-gene regulations. We also compare the reliable GINs with those GINs reconstructed by a global proximity measure Pearson correlation coefficient method from single datasets. It has been demonstrated that our reliable GINs achieve much better prediction performance especially with much higher precision. The functional enrichment analysis also suggests that gene sets in a reliable GIN are more functionally significant. Our method is especially useful to decipher GINs from multiple TCGx datasets related to less studied organisms where little knowledge is available except gene expression data.

Establishing causal relations between random variables from observational data is perhaps the most important challenge in today's science. In remote sensing and geosciences, this is of special relevance to better understand the earth's system and the complex interactions between the governing processes. In this paper, we focus on an observational causal inference, and thus, we try to estimate the correct direction of causation using a finite set of empirical data. In addition, we focus on the more complex bivariate scenario that requires strong assumptions and no conditional independence tests can be used. In particular, we explore the framework of (nondeterministic) additive noise models, which relies on the principle of independence between the cause and the generating mechanism. A practical algorithmic instantiation of such principle only requires: 1) two regression models in the forward and backward directions and 2) the estimation of statistical independence between the obtained residuals and the observations. The direction leading to more independent residuals is decided to be the cause. We instead propose a criterion that uses the sensitivity (derivative) of the dependence estimator, the sensitivity criterion allows to identify samples most affecting the dependence measure, and hence, the criterion is robust to spurious detections. We illustrate the performance in a collection of 28 geoscience causal inference problems, a database of radiative transfer models simulations and machine learning emulators in vegetation parameter modeling involving 182 problems, and assessing the impact of different regression models in a carbon cycle problem. The criterion achieves the state-of-the-art detection rates in all cases, and it is generally robust to noise sources and distortions. The presented approach confirms the validity in observational bivariate problems in the earth sciences.

Bayesian networks are formal knowledge representation tools that provide reasoning under uncertainty. The applications of Bayesian networks are widespread, including data mining, information retrieval, and various diagnostic systems. Although Bayesian networks are useful, the learning problem, namely to construct a network automatically from data, remains a difficult problem. Recently, some researchers have adopted evolutionary computation for learning. However, the drawback is that the approach is slow. In this chapter, we propose a hybrid framework for Bayesian network learning. By combining the merits of two different learning approaches, we expect an improvement in learning speed. In brief, the new learning algorithm consists of two phases: the conditional independence (CI) test phase and the search phase. In the CI test phase, we conduct dependency analysis, which helps to reduce the search space. In the search phase, we perform model searching using an evolutionary approach, called cooperative coevolution. When comparing our new algorithm with an existing algorithm, we find that our algorithm performs faster and is more accurate in many cases.