Fast Markov Blanket Discovery Without Causal Sufficiency

Towards efficient and effective discovery of Markov blankets for feature selection

The Markov blanket (MB) in Bayesian networks has attracted much attention since the MB of a target attribute (T) is the minimal feature subset with maximum prediction ability for classification. Nevertheless, traditional MB discovery methods such as IAMB, HITON-MB, and MMMB are not suitable for streaming features. Meanwhile, online feature selection with streaming features (OSFSF) methods such as Alpha-investing and SAOLA, focus only on the relevance and ignores causality of the features, they cannot mine the MB, or only find the parents and children (PC) such as OSFS. Therefore, these methods have weaker interpretability and do not have sufficient prediction accuracy. We propose a novel algorithm for online markov blanket discovery with streaming features to tackle abovementioned issues, named OMBSF. OMBSF finds the MB of T containing parents, children, and spouses, discards false positives from the PC and spouse sets online, distinguishes between the PC and spouse in real-time. An empirical study demonstrates that OMBSF finds a more accurate MB when the volume of features is high and it significantly improves the prediction accuracy than other algorithms. Moreover, OMBSF obtains a bigger feature subset than that obtained by OSFS, demonstrating that OMBSF can identify the spouse in the MB that are not identified using OSFS.

Based on benchmarking various methods and analyzing multiple real scRNA-seq datasets, a computational platform/workflow and a set of tips for best practices are developed for analyzing causal interactions or relationships in single cells.

Unsupervised feature selection based on Markov blanket and particle swarm optimization

In faithful Bayesian networks, the Markov blanket of the class attribute is a unique and minimal feature subset for optimal feature selection. However, little attention has been paid to Markov blanket feature selection in a non-faithful environment which widely exists in the real world. To tackle this issue, in this paper, we deal with non-faithful data distributions and propose the concept of representative sets instead of Markov blankets. With a standard sparse group lasso for selection of features from the representative sets, we design an effective algorithm, SRS, for Markov blanket feature Selection via Representative Sets with non-faithful data distributions. Empirical studies demonstrate that SRS outperforms the state-of-the-art Markov blanket feature selectors and other well-established feature selection methods.

We present an algorithmic framework for learning local causal structure around target variables of interest in the form of direct causes/effects and Markov blankets applicable to very large data sets with relatively small samples. The selected feature sets can be used for causal discovery and classification. The framework (Generalized Local Learning, or GLL) can be instantiated in numerous ways, giving rise to both existing state-of-the-art as well as novel algorithms. The resulting algorithms are sound under well-defined sufficient conditions. In a first set of experiments we evaluate several algorithms derived from this framework in terms of predictivity and feature set parsimony and compare to other local causal discovery methods and to state-of-the-art non-causal feature selection methods using real data. A second set of experimental evaluations compares the algorithms in terms of ability to induce local causal neighborhoods using simulated and resimulated data and examines the relation of predictivity with causal induction performance. Our experiments demonstrate, consistently with causal feature selection theory, that local causal feature selection methods (under broad assumptions encompassing appropriate family of distributions, types of classifiers, and loss functions) exhibit strong feature set parsimony, high predictivity and local causal interpretability. Although non-causal feature selection methods are often used in practice to shed light on causal relationships, we find that they cannot be interpreted causally even when they achieve excellent predictivity. Therefore we conclude that only local causal techniques should be used when insight into causal structure is sought. In a companion paper we examine in depth the behavior of GLL algorithms, provide extensions, and show how local techniques can be used for scalable and accurate global causal graph learning.

Online feature selection for multi-source streaming features

The K2 algorithm is widely recognised as an effective approach for Bayesian network structure learning, but its performance is highly sensitive to the chosen node order. To mitigate this limitation, we introduce a new method for deriving candidate parents that serve as input to the K2 algorithm. Our approach begins by estimating the Markov blanket of each variable using an elastic net–penalised Markov blanket procedure. We then employ a dependency-based scoring function to identify candidate parents within each Markov blanket. These selected parents are subsequently provided to the K2 algorithm to learn the network structure. Experimental results across multiple datasets show that the proposed method reduces unnecessary edges and achieves substantially better performance than existing approaches.

Causal discovery and causal effect estimation are two fundamental tasks in causal inference. While many methods have been developed for each task individually, statistical challenges arise when applying these methods jointly: estimating causal effects after running causal discovery algorithms on the same data leads to “double dipping,” invalidating the coverage guarantees of classical confidence intervals. To this end, we develop tools for valid post-causal-discovery inference. Across empirical studies, we show that a naive combination of causal discovery and subsequent inference algorithms leads to highly inflated miscoverage rates; on the other hand, applying our method provides reliable coverage while allowing for a trade-off between causal discovery accuracy and confidence interval width. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Feature selection has achieved unprecedented success in obtaining sparse discriminative features. However, the existing methods almost use the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,p}$</tex-math></inline-formula> -norm constraint on transformation matrix to obtain sparse features, which introduces extra parameters and cannot obtain the features directly. In addition, existing algorithms only focused on the global structure and ignored the local structure, leading to poor performance when solving data with non-Gaussian distributions which a single center point cannot describe precisely. Based on above considerations, we propose a supervised feature selection via multi-center and local structure learning. We further introduce trace ratio criterion into our model in favor of improving the discriminant of features selected. In order to address the overlap problem, we use multiple center points to match the distribution of data and construct a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -Nearest Neighbor graph to explore the local structure of the data. In addition, we also propose an efficient method to optimize the transformation matrix with the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math></inline-formula> -norm constraint and can directly obtain the sparse features. We evaluate our method on Toy datasets and several real-world datasets, show improvement over state-of-the-art feature selection methods, and demonstrate the effectiveness of our model in dealing with non-Gaussian distributed data problems.

A new filter feature selection algorithm for classification task by ensembling pearson correlation coefficient and mutual information

Toward Explainable and Sustainable Manufacturing: Benchmarking Causal Discovery on Process Duration and Energy Use

Aims: Accurately identifying diagnostic biomarkers for Autism Spectrum Disorder (ASD) is crucial for enabling early diagnosis and timely intervention. Brain causal networks, which outline causal relationships and information transmission pathways among different brain regions, hold significant diagnostic value for ASD. Identifying ASD-relevant causal relationships between brain regions is critical for ASD diagnosis and for elucidating its pathophysiology. This study proposes a new model called the Peter and Clark Momentary Conditional Independence (PCMCI)-Support Vector Machine (SVM) model, which can identify diagnostic biomarkers for ASD. Methods: The brain blood oxygen level-dependent signals of 116 brain regions from 167 participants, consisting of 72 participants with ASD and 95 Typical Controls (TCs), were used to derive brain causal networks by detecting inter-regional causal relationships using five causal discovery algorithms; the PCMCI algorithm, spectral dynamic causal modeling, Granger causal analysis, Transfer Entropy, and Liang-Kleeman information flow causal analysis. Then, the brain causal relationships were fed into the SVM, Random Forest (RF), and K-Nearest Neighbors (KNN) classifiers, respectively to classify individuals with ASD and TCs, and thereby obtain a model suitable for identifying diagnostic biomarkers for ASD through causal network analysis. Results: Experimental results demonstrate that the PCMCI-SVM model achieves an accuracy of 91.29% in ASD identification and outperforms the other four causal discovery algorithms, as well as the RF and KNN classifiers. Moreover, our analysis indicates that the right thalamus and right middle temporal gyrus are potential diagnostic biomarkers for ASD. Additionally, the causal relationship between [left inferior parietal→right insula] was found to be associated with the dorsal attention network, while [right cuneus→left cuneus] was associated with the visual network, suggesting that disruptions in these causal relationships may impair the functional integrity of their respective subnetworks. Conclusion: Our findings show that cognitive processes and brain connectivity are largely influenced by causal interactions between different brain regions. These potential diagnostic biomarkers not only offer insights into the neurofunctional mechanisms of ASD but also hold promise for improving diagnostic accuracy for ASD.

The common interpretation of Bayesian networks is that they are vehicles for representing probability distributions, in a graphical form supportive of human understanding and with computational mechanisms supportive of probabilistic reasoning (updating). But the interpretation of Bayesian networks assumed by causal discovery algorithms is causal: the links in the graphs specifically represent direct causal connections between variables. However, there is some tension between these two interpretations. The philosophy of probabilistic causation posits a particular connection between the two, namely that causal relations of certain kinds give rise to probabilistic relations of certain kinds. Causal discovery algorithms take advantage of this kind of connection by ruling out some Bayesian networks given observational data not supported by the posited probability-causality relation. But the discovered (remaining) Bayesian networks are then specifically causal, and not simply arbitrary representations of probability.There are multiple contentious issues underlying any causal interpretation of Bayesian networks. We will address the following questions: Since Bayesian net construction rules allow the construction of multiple distinct networks to represent the very same probability distribution, how can we come to prefer any specific one as “the” causal network? Since Bayesian nets within a Verma-Pearl pattern are strongly indistinguishable, how can causal discovery ever come to select exactly one network as “the” causal network? Causal discovery assumes faithfulness (that d-connections in the model are accompanied by probabilistic dependency in the system modeled). However, some physical systems cannot be modeled faithfully under a causal interpretation. How can causal discovery cope with that? Here we introduce a causal interpretation of Bayesian networks by way of answering these questions and then apply this interpretation to answering further questions about causal power, explanation and responsibility.Keywordscausal discoveryfaithfulnessBayesian networksprobabilistic causalityinterventioncausal powercausal responsibility

Dimensionality reduction of attribute set is a common preprocessing step used in machine learning. This step is especially important for high-dimensional data with low-dimensional representation such as gene expression data. Feature reduction is essential in the case of microarray data because most of the microarray data attributes are believed to be unrelated to observed classes. This paper proposes a three-step feature selection framework based on feature clustering, multi-criteria assessment (Borda count) and Markov blanket. The proposed framework is a filter method so it can be used with any classification algorithm. Its classification performance and selection stability were assessed. The experimental studies were performed on 10 microarray data sets. The experimental evaluation showed that the Markov blanket filter produces results comparable to state-of-art methods in terms of classification performance. However it tends to produce unstable solutions.