FastGGM: An Efficient Algorithm for the Inference of Gaussian Graphical Model in Biological Networks.

Gaussian Graphical models (GGM) are widely used to estimate network structure in domains ranging from biology to finance. In practice, data is often corrupted by latent confounders which biases inference of the underlying true graphical structure. In this paper, we compare and contrast two strategies for inference in graphical models with latent confounders: Gaussian graphical models with latent variables (LVGGM) and PCA-based removal of confounding (PCA+GGM). While these two approaches have similar goals, they are motivated by different assumptions about confounding. In this paper, we explore the connection between these two approaches and propose a new method, which combines the strengths of these two approaches. We prove the consistency and convergence rate for the PCA-based method and use these results to provide guidance about when to use each method. We demonstrate the effectiveness of our methodology using both simulations and two real-world applications.

For many complex heterogeneous diseases, the underlying mechanisms contributing to pathogenesis is still unknown. Although prediction models for complex heterogeneous diseases have been trained on clinical features and imaging data, they may not reveal mechanisms contributing to pathogenesis. The development of human diseases often involves a highly interactive and dynamic system of molecular layers. Networks are a useful framework for representing cross-talk relationships of molecules in biological and disease pathways. Understanding complex biological networks is a fundamental and challenging issue in computational and systems biology. Constructing disease-specific networks is still lacking in many diseases even with current advances. It necessitates the development of methods to integrate protein and genetic interaction information from knowledgebases with omics data to reconstruct networks and develop predictive models to effectively integrate different omics data and network information while being interpretable. Biological networks are often inferred through Gaussian Graphical Models (GGMs) using gene or protein expression data only. GGMs identify conditional dependence by estimating a precision matrix between genes or proteins. However, conventional GGM approaches often ignore prior knowledge about protein-protein interactions (PPI). Recently, several groups have extended GGM to incorporate PPI information. However, these methods are either computationally intractable for large-scale data, or disregard weights in the PPI networks. To address these shortcomings, we developed an augmented high-dimensional graphical lasso (AhGlasso) method to incorporate edge weights in known protein-protein interactions with omics data for global network learning. This new method outperforms weighted graphical lasso-based algorithms with respect to computational time in simulated large-scale data settings while achieving better or comparable prediction accuracy of node connections. Using proteomic data from a study on chronic obstructive pulmonary disease (COPD), we demonstrated that AhGlasso improves protein network inference compared to other approaches that incorporate PPI information. Recently Deep Learning including convolution Graph Neural Network (ConvGNN) has shown great potential for disease classification by integrating a known PPI network from existing databases and a single omics data set. We extended the existing ConvGNN framework to incorporate PPI in addition to multi-omics data to develop a prediction model for COPD. This model achieved significantly better accuracy compared to several conventional classification methods. We also demonstrated that the updated COPD-associated network with AhGlasso further improves the prediction accuracy. Moreover, we applied explainable machine learning methods for interpretation and identified significant genes/proteins and important subnetworks in the ConvGNN model for COPD prediction. Our study suggests the benefits of using ConvGNN for the prediction of heterogeneous complex diseases by integrating multi-omics data and interaction networks.

The Gaussian graphical model (GGM) is a powerful tool to describe the relationship between the nodes via the inverse of the covariance matrix in a complex biological system. But the inference of this matrix is problematic because of its high dimension and sparsity. From previous analyses, it has been shown that the Bernstein and Szasz polynomials can improve the accuracy of the estimate if they are used in advance of the inference as a processing step of the data. Hereby in this study, we consider whether any type of the Bernstein operators such as the Bleiman Butzer Hahn, Meyer-Konig, and Zeller operators can be performed for the improvement of the accuracy or only the Bernstein and the Szasz polynomials can satisfy this condition. From the findings of the Monte Carlo runs, we detect that the highest accuracies in GGM can be obtained under the Bernstein and Szasz polynomials, rather than all other types of the Bernstein polynomials, from small to high-dimensional biological networks.

Differential network analysis investigates how the network of connected genes changes from one condition to another and has become a prevalent tool to provide a deeper and more comprehensive understanding of the molecular etiology of complex diseases. Based on the asymptotically normal estimation of large Gaussian graphical model (GGM) in the high-dimensional setting, we developed a computationally efficient test for differential network analysis through testing the equality of two precision matrices, which summarize the conditional dependence network structures of the genes. Additionally, we applied a multiple testing procedure to infer the differential network structure with false discovery rate (FDR) control. Through extensive simulation studies with different combinations of parameters including sample size, number of vertices, level of heterogeneity and graph structure, we demonstrated that our method performed much better than the current available methods in terms of accuracy and computational time. In real data analysis on lung adenocarcinoma, we revealed a differential network with 3503 nodes and 2550 edges, which consisted of 50 clusters with an FDR threshold at 0.05. Many of the top gene pairs in the differential network have been reported relevant to human cancers. Our method represents a powerful tool of network analysis for high-dimensional biological data.

Extended message passing algorithm for inference in loopy Gaussian graphical models

Constructing expression networks using transcriptomic data is an effective approach for studying gene regulation. A popular approach for constructing such a network is based on the Gaussian graphical model (GGM), in which an edge between a pair of genes indicates that the expression levels of these two genes are conditionally dependent, given the expression levels of all other genes. However, GGMs are not appropriate for non-Gaussian data, such as those generated in RNA-seq experiments. We propose a novel statistical framework that maximizes a penalized likelihood, in which the observed count data follow a Poisson log-normal distribution. To overcome the computational challenges, we use Laplace's method to approximate the likelihood and its gradients, and apply the alternating directions method of multipliers to find the penalized maximum likelihood estimates. The proposed method is evaluated and compared with GGMs using both simulated and real RNA-seq data. The proposed method shows improved performance in detecting edges that represent covarying pairs of genes, particularly for edges connecting low-abundant genes and edges around regulatory hubs.

Loss function, unbiasedness, and optimality of Gaussian graphical model selection

The multivariate adaptive regression spline (MARS) is one of the alternative nonparametric approaches of the Gaussian graphical model (GGM) in order to construct the undirected networks of complex biological systems. In GGM, it is assumed that the binary form of the estimated precision matrix, computed from the systems' elements under their conditional independencies, represents the undirected interactions of these elements by using the lasso regression. From previous studies, it has been shown that if MARS is estimated by discarding all interactions and nonlinear components, resulting in a sole inference of the main effects, it can be a strong alternative of GGM. We call this type of the summarized MARS model the lasso-based MARS model. Furthermore, in the construction of the networks via the optimal MARS model, it is assumed that the estimated regression coefficients, i.e., the important terms of the estimated model, can be the indication of the pairwise links between the species of the systems. Hereby, in this study, we investigate whether the inference of the networks via the Pearson, Spearman, and Kendall's tau correlation coefficients has better performance than the networks constructed by the important regression coefficients in the lasso-based MARS model. In the analyses, we use both simulated and real datasets and compare the estimated networks by various accuracy measures from specificity and F-measure to Matthews correlation coefficient. From the analyses, we find that the suggested approaches improve all the listed criteria significantly without any computational cost.

Attaining a detailed understanding of the various biological networks in an organism lies at the core of the emerging discipline of systems biology. A precise description of the relationships formed between genes, mRNA molecules, and proteins is a necessary step toward a complete description of the dynamic behavior of an organism at the cellular level, and toward intelligent, efficient, and directed modification of an organism. The importance of understanding such regulatory, signaling, and interaction networks has fueled the development of numerous in silico inference algorithms, as well as new experimental techniques and a growing collection of public databases. The Software Environment for BIological Network Inference (SEBINI) has been created to provide an interactive environment for the deployment, evaluation, and improvement of algorithms used to reconstruct the structure of biological regulatory and interaction networks. SEBINI can be used to analyze high-throughput gene expression, protein abundance, or protein activation data via a suite of state-of-the-art network inference algorithms. It also allows algorithm developers to compare and train network inference methods on artificial networks and simulated gene expression perturbation data. SEBINI can therefore be used by software developers wishing to evaluate, refine, or combine inference techniques, as well as by bioinformaticians analyzing experimental data. Networks inferred from the SEBINI software platform can be further analyzed using the Collective Analysis of Biological Interaction Networks (CABIN) tool, which is an exploratory data analysis software that enables integration and analysis of protein-protein interaction and gene-to-gene regulatory evidence obtained from multiple sources. The collection of edges in a public database, along with the confidence held in each edge (if available), can be fed into CABIN as one "evidence network," using the Cytoscape SIF file format. Using CABIN, one may increase the confidence in individual edges in a network inferred by an algorithm in SEBINI, as well as extend such a network by combining it with species-specific or generic information, e.g., known protein-protein interactions or target genes identified for known transcription factors. Thus, the combined SEBINI-CABIN toolkit aids in the more accurate reconstruction of biological networks, with less effort, in less time.A demonstration web site for SEBINI can be accessed from https://www.emsl.pnl.gov/SEBINI/RootServlet . Source code and PostgreSQL database schema are available under open source license. ronald.taylor@pnl.gov. For commercial use, some algorithms included in SEBINI require licensing from the original developers. CABIN can be downloaded from http://www.sysbio.org/dataresources/cabin.stm . mudita.singhal@pnl.gov.

It is well known that regression methods designed for clean data will lead to erroneous results if directly applied to corrupted data. Despite the recent methodological and algorithmic advances in Gaussian graphical model estimation, how to achieve efficient and scalable estimation under contaminated covariates is unclear. Here a new methodology called convex conditioned innovative scalable efficient estimation (COCOISEE) for Gaussian graphical model under both additive and multiplicative measurement errors is developed. It combines the strengths of the innovative scalable efficient estimation in Gaussian graphical model and the nearest positive semi-definite matrix projection, thus enjoying stepwise convexity and scalability. Comprehensive theoretical guarantees are provided and the effectiveness of the proposed methodology is demonstrated through numerical studies.

This doctoral thesis contributes to the advancement of statistical analyses of gene expression profiles using Gaussian graphical models (GGMs). The main focus has been on studying the properties of GGMs obtained with the LW shrinkage. The premise is that, despite its advantages, the shrinkage approach introduces some biases that have not been studied in sufficient detail. These biases have the potential of obscuring the interpretation of the network structure and impeding the validation of earlier analyses. In this sense, this thesis may be considered a continuation of the works of Schäfer and Strimmer, where the LW-shrinkage was originally used to model GRNs with GGMs. The thesis is organized as follows; Chapter 1 presents a test of statistical significance for GGMs based on the LW-shrinkage. Here the probability density of the ‘shrunk’ partial correlation is derived by means of geometric arguments. In Chapter 2 a network analysis of (matched) nasal and bronchial expression profiles is presented. The method developed in the previous chapter is employed to explore whether expression profiles from nasal epithelial cells can be used as a proxy for bronchial epithelial cells. Chapter 3 shows the existence of a non-linear bias on the partial correlations obtained with the LW-shrinkage. This bias is removed via ‘un-shrinking’; a new concept that de-regularizes the partial correlation. In Chapter 4 the LW-shrinkage is revisited from a data-level perspective. The goal is to explore the correspondence between the shrinkage-based covariance matrix and the dataset.

Nitrogen is an essential nutrient for all life forms. Like most unicellular organisms, the yeast Saccharomyces cerevisiae transports and catabolizes good nitrogen sources in preference to poor ones. Nitrogen catabolite repression (NCR) refers to this selection mechanism. We propose an approach based on Gaussian graphical models (GGMs), which enable to distinguish direct from indirect interactions between genes, to identify putative NCR genes from putative NCR regulatory motifs and over-represented motifs in the upstream noncoding sequences of annotated NCR genes. Because of the high-dimensionality of the data, we use a shrinkage estimator of the covariance matrix to infer the GGMs. We show that our approach makes significant and biologically valid predictions. We also show that GGMs are more effective than models that rely on measures of direct interactions between genes.

Gene regulatory networks (GRNs) are often inferred based on Gaussian graphical models that could identify the conditional dependence among genes by estimating the corresponding precision matrix. Classical Gaussian graphical models are usually designed for single network estimation and ignore existing knowledge such as pathway information. Therefore, they can neither make use of the common information shared by multiple networks, nor can they utilize useful prior information to guide the estimation. In this paper, we propose a new weighted fused pathway graphical lasso (WFPGL) to jointly estimate multiple networks by incorporating prior knowledge derived from known pathways and gene interactions. Based on the assumption that two genes are less likely to be connected if they do not participate together in any pathways, a pathway-based constraint is considered in our model. Moreover, we introduce a weighted fused lasso penalty in our model to take into account prior gene interaction data and common information shared by multiple networks. Our model is optimized based on the alternating direction method of multipliers (ADMM). Experiments on synthetic data demonstrate that our method outperforms other five state-of-the-art graphical models. We then apply our model to two real datasets. Hub genes in our identified state-specific networks show some shared and specific patterns, which indicates the efficiency of our model in revealing the underlying mechanisms of complex diseases.

This paper concerns the specification, and performance, of scale-free prior distributions with a view toward large-scale network inference from small-sample data sets. We devise three scale-free priors and implement them in the framework of Gaussian graphical models. Gaussian graphical models are used in gene network inference where high-throughput data describing a large number of variables with comparatively few samples are frequently analyzed by practitioners. And, although there is a consensus that many such networks are scale-free, the modus operandi is to assign a random network prior. Simulations demonstrate that the scale-free priors outperform the random network prior at recovering scale-free trees with degree exponents near 2, such as are characteristic of many real-world systems. On the other hand, the random network prior compares favorably at recovering scale-free trees characterized by larger degree exponents.

We propose a message propagation scheme for numerically stable inference in Gaussian graphical models which can otherwise be susceptible to errors caused by finite numerical precision. We adapt square root algorithms, popular in Kalman filtering, to graphs with arbitrary topologies. The method consists of maintaining potentials and generating messages that involve the square root of precision matrices. Combining this with the machinery of the junction tree algorithm leads to an efficient and numerically stable algorithm. Experiments are presented to demonstrate the robustness of the method to numerical errors that can arise in complex learning and inference problems.