Gene Networks Based on the Graphical Gaussian Model

This doctoral thesis contributes to the advancement of statistical analyses of gene expression profiles using Gaussian graphical models (GGMs). The main focus has been on studying the properties of GGMs obtained with the LW shrinkage. The premise is that, despite its advantages, the shrinkage approach introduces some biases that have not been studied in sufficient detail. These biases have the potential of obscuring the interpretation of the network structure and impeding the validation of earlier analyses. In this sense, this thesis may be considered a continuation of the works of Schäfer and Strimmer, where the LW-shrinkage was originally used to model GRNs with GGMs. The thesis is organized as follows; Chapter 1 presents a test of statistical significance for GGMs based on the LW-shrinkage. Here the probability density of the ‘shrunk’ partial correlation is derived by means of geometric arguments. In Chapter 2 a network analysis of (matched) nasal and bronchial expression profiles is presented. The method developed in the previous chapter is employed to explore whether expression profiles from nasal epithelial cells can be used as a proxy for bronchial epithelial cells. Chapter 3 shows the existence of a non-linear bias on the partial correlations obtained with the LW-shrinkage. This bias is removed via ‘un-shrinking’; a new concept that de-regularizes the partial correlation. In Chapter 4 the LW-shrinkage is revisited from a data-level perspective. The goal is to explore the correspondence between the shrinkage-based covariance matrix and the dataset.

Gaussian Graphical Models (GGMs) are extensively used in many research areas, such as genomics, proteomics, neuroimaging, and psychology, to study the partial correlation structure of a set of variables. This structure is visualized by drawing an undirected network, in which the variables constitute the nodes and the partial correlations the edges. In many applications, it makes sense to impose sparsity (i.e., some of the partial correlations are forced to zero) as sparsity is theoretically meaningful and/or because it improves the predictive accuracy of the fitted model. However, as we will show by means of extensive simulations, state-of-the-art estimation approaches for imposing sparsity on GGMs, such as the Graphical lasso, ℓ1 regularized nodewise regression, and joint sparse regression, fall short because they often yield too many false positives (i.e., partial correlations that are not properly set to zero). In this paper we present a new estimation approach that allows to control the false positive rate better. Our approach consists of two steps: First, we estimate an undirected network using one of the three state-of-the-art estimation approaches. Second, we try to detect the false positives, by flagging the partial correlations that are smaller in absolute value than a given threshold, which is determined through cross-validation; the flagged correlations are set to zero. Applying this new approach to the same simulated data, shows that it indeed performs better. We also illustrate our approach by using it to estimate (1) a gene regulatory network for breast cancer data, (2) a symptom network of patients with a diagnosis within the nonaffective psychotic spectrum and (3) a symptom network of patients with PTSD.

A major challenge in contemporary neuroscience is to analyze data from large numbers of neurons recorded simultaneously across many experimental replications (trials), where the data are counts of neural firing events, and one of the basic problems is to characterize the dependence structure among such multivariate counts. Methods of estimating high-dimensional covariation based on ℓ 1-regularization are most appropriate when there are a small number of relatively large partial correlations, but in neural data there are often large numbers of relatively small partial correlations. Furthermore, the variation across trials is often confounded by Poisson-like variation within trials. To overcome these problems we introduce a comprehensive methodology that imbeds a Gaussian graphical model into a hierarchical structure: the counts are assumed Poisson, conditionally on latent variables that follow a Gaussian graphical model, and the graphical model parameters, in turn, are assumed to depend on physiologically-motivated covariates, which can greatly improve correct detection of interactions (non-zero partial correlations). We develop a Bayesian approach to fitting this covariate-adjusted generalized graphical model and we demonstrate its success in simulation studies. We then apply it to data from an experiment on visual attention, where we assess functional interactions between neurons recorded from two brain areas.

This paper introduces a MATLAB toolbox, MGraph, which applies graphical models as a natural environment to formulate and solve problems in microarray data analysis. MGraph with its graphical interface allows the user to predict genetic regulatory networks by a graphical gaussian model (GGM), and to quantify the effects of different experimental treatment conditions on gene expression profiles by a graphical log-linear model (GLM). The power of graphical models was explored and illustrated through two example applications. First, four MAPK pathways in yeast were meaningfully reconstructed through GGM. Second, GLM was used to quantify the contributions of sex, genotype and age to transcriptional variance in Drosophila melanogaster. This application may provide a valuable aid in the prediction of genetic regulatory networks, as well as in investigations of various experimental conditions that affect global gene expression profiles. The MATLAB program MGraph is freely available at http://www.uio.no/~junbaiw/mgraph/mgraph.html for academics.

Gaussian graphical models (GGM) are powerful tools to examine partial correlation structures in high-dimensional omics datasets. Partial correlation networks can explain complex relationships between genes or other biological variables. Bayesian implementations of GGMs have recently received more attention. Usually, the most demanding parts of GGM implementations are: (i) hyperparameter tuning, (ii) edge selection, (iii) scalability for large datasets, and (iv) the prior choice for Bayesian GGM. To address these limitations, we introduce a novel Bayesian GGM using a hierarchical matrix-F prior with a fast implementation. We show, with extensive simulations and biological example analyses, that this prior has competitive network recovery capabilities compared to state-of-the-art approaches and good properties for recovering meaningful networks. We present a new way of tuning the shrinkage hyperparameter by constraining the condition number of the estimated precision matrix. For edge selection, we propose using approximated credible intervals (CI) whose width is controlled by the false discovery rate. An optimal CI is selected by maximizing an estimated F1-score via permutations. In addition, a specific choice of hyperparameter can make the proposed prior better suited for clustering and community detection. Our method, with a generalized expectation-maximization algorithm, computationally outperforms existing Bayesian GGM approaches that use Markov chain Monte Carlo algorithms. The method is implemented in the R package HMFGraph, found on GitHub at https://github.com/AapoKorhonen/HMFGraph. All codes to reproduce the results are found on GitHub at https://github.com/AapoKorhonen/HMFGraph-Supplementary.

Clustering genes with high correlations will group genes with close expression profiles, defining clusters of co-expressed genes. However, such correlations do not provide any clue on the chain of information going from gene to gene. Partial correlation consists in quantifying the correlation between two genes after excluding the effects of the other genes. Partial correlation thus makes it possible to distinguish between the correlation of two genes due to direct causal relationships from the correlation that originates via intermediate genes. In this chapter, Gaussian graphical model (GGM) learning is set up as a covariate selection problem. Two least absolute shrinkage and selection operator (LASSO)-type techniques are described, the graphical LASSO approach and the neighborhood selection. Then two extensions to the classical GGM are presented. GGMs are extended in structured GGMs, to account for modularity, and more generally heterogeneity in the gene connection features. The extension using a biological prior on the network structure is illustrated on real data.

The multivariate adaptive regression spline (MARS) is one of the alternative nonparametric approaches of the Gaussian graphical model (GGM) in order to construct the undirected networks of complex biological systems. In GGM, it is assumed that the binary form of the estimated precision matrix, computed from the systems' elements under their conditional independencies, represents the undirected interactions of these elements by using the lasso regression. From previous studies, it has been shown that if MARS is estimated by discarding all interactions and nonlinear components, resulting in a sole inference of the main effects, it can be a strong alternative of GGM. We call this type of the summarized MARS model the lasso-based MARS model. Furthermore, in the construction of the networks via the optimal MARS model, it is assumed that the estimated regression coefficients, i.e., the important terms of the estimated model, can be the indication of the pairwise links between the species of the systems. Hereby, in this study, we investigate whether the inference of the networks via the Pearson, Spearman, and Kendall's tau correlation coefficients has better performance than the networks constructed by the important regression coefficients in the lasso-based MARS model. In the analyses, we use both simulated and real datasets and compare the estimated networks by various accuracy measures from specificity and F-measure to Matthews correlation coefficient. From the analyses, we find that the suggested approaches improve all the listed criteria significantly without any computational cost.

In this article, we use partial correlations to derive bi‐directional connections between major firms listed in the Moscow Stock Exchange. We obtain coefficients of partial correlation from the correlation estimates of the Constant Conditional Correlation GARCH (CCC‐GARCH) and the consistent Dynamic Conditional Correlation GARCH (cDCC‐GARCH) models. We map the graph of partial correlations using the Gaussian Graphical Model and apply network analysis to identify the most central firms in terms of both shock propagation and connectedness with others. Moreover we analyze some network characteristics over time and based on these we construct a measure of system vulnerability to external shocks. Our findings suggest that during the crisis interconnectedness between firms strengthens and becomes polarized and the system becomes more vulnerable to systemic shocks. In addition, we found that the most connected firms are the state‐owned firms Sberbank and Gazprom and the private oil company Lukoil, while in terms of the top most central systemic risk contributors, Sberbank gave its place to the NLMK Group.

In this paper we derive the asymptotic posterior distribution, in a conjugate analysis, for the marginal and partial correlation coefficients in a graphical Gaussian model. An example of prior to posterior analysis is given and the problem of the specification of the hyper parameters discussed.

BackgroundGraphical Gaussian models are popular tools for the estimation of (undirected) gene association networks from microarray data. A key issue when the number of variables greatly exceeds the number of samples is the estimation of the matrix of partial correlations. Since the (Moore-Penrose) inverse of the sample covariance matrix leads to poor estimates in this scenario, standard methods are inappropriate and adequate regularization techniques are needed. Popular approaches include biased estimates of the covariance matrix and high-dimensional regression schemes, such as the Lasso and Partial Least Squares.ResultsIn this article, we investigate a general framework for combining regularized regression methods with the estimation of Graphical Gaussian models. This framework includes various existing methods as well as two new approaches based on ridge regression and adaptive lasso, respectively. These methods are extensively compared both qualitatively and quantitatively within a simulation study and through an application to six diverse real data sets. In addition, all proposed algorithms are implemented in the R package "parcor", available from the R repository CRAN.ConclusionIn our simulation studies, the investigated non-sparse regression methods, i.e. Ridge Regression and Partial Least Squares, exhibit rather conservative behavior when combined with (local) false discovery rate multiple testing in order to decide whether or not an edge is present in the network. For networks with higher densities, the difference in performance of the methods decreases. For sparse networks, we confirm the Lasso's well known tendency towards selecting too many edges, whereas the two-stage adaptive Lasso is an interesting alternative that provides sparser solutions. In our simulations, both sparse and non-sparse methods are able to reconstruct networks with cluster structures. On six real data sets, we also clearly distinguish the results obtained using the non-sparse methods and those obtained using the sparse methods where specification of the regularization parameter automatically means model selection. In five out of six data sets, Partial Least Squares selects very dense networks. Furthermore, for data that violate the assumption of uncorrelated observations (due to replications), the Lasso and the adaptive Lasso yield very complex structures, indicating that they might not be suited under these conditions. The shrinkage approach is more stable than the regression based approaches when using subsampling.

Loss function, unbiasedness, and optimality of Gaussian graphical model selection

Research background: Every real estate may be described with a multitude of attributes. In the process of real estate appraisal only those properties are taken into account that significantly affect its value. Mass appraisal involves a simultaneous valuation of many similar real properties, carried out in the same manner and at the same time. The algorithm applied to mass appraisal ought to ensure a uniform approach to the valuation of all real estate of the same type in an objective fashion. Purpose: The purpose of the paper is to define the weights of attributes in the process of real estate mass appraisal on the basis of relationships between unit property value and the values of attributes. Research methodology: The weights were defined on the grounds of partial correlation coefficients for the qualitative properties (Spearman rank correlation coefficient and τB Kendall correlation coefficient). Results: The signs of certain correlation coefficients were discordant with the actual direction of the relations between the analysed properties. The problem was avoided by employing partial correlation coefficients. On the basis of the calculated partial correlation coefficients the weights of individual attributes were calculated. Of all analysed coefficients the partial τB Kendall correlation coefficient is methodologically the most suitable one. Novelty: The use of partial correlation coefficients for determining attribute weights is an innovative approach and is applied in the article.

Graphical models provide a framework for exploration of multivariate dependence patterns. The connection between graph and statistical model is made by identifying the vertices of the graph with the observed variables and translating the pattern of edges in the graph into a pattern of conditional independences that is imposed on the variables’ joint distribution. Focusing on Gaussian models, we review classical graphical models. For these models the defining conditional independences are equivalent to vanishing of certain (partial) correlation coefficients associated with individual edges that are absent from the graph. Hence, Gaussian graphical model selection can be performed by multiple testing of hypotheses about vanishing (partial) correlation coefficients. We show and exemplify how this approach allows one to perform model selection while controlling error rates for incorrect edge inclusion.

Network reconstruction is a main goal of many biological endeavors. Graphical Gaussian models (GGMs) are often used since the underlying assumptions are well understood, the graph is readily estimated by calculating the partial correlation (paCor) matrix, and its interpretation is straightforward. In spite of these advantages, GGMs are limited in that interactions are not accommodated as the underlying multivariate normality assumption allows for linear dependencies only. As we show, when applied in the presence of interactions, the GGM framework can lead to incorrect inference regarding dependence. Identifying the exact dependence structure in this context is a difficult problem, largely because an analogue of the paCor matrix is not available and dependencies can involve many nodes. We here present a computationally efficient approach to identify bivariate interactions in networks. A key element is recognizing that interactions have a marginal linear effect and as a result information about their presence can be obtained from the paCor matrix. Theoretical derivations for the exact effect are presented and used to motivate the approach; and simulations suggest that the method works well, even in fairly complicated scenarios. Practical advantages are demonstrated in analyses of data from a breast cancer study.

Gaussian Graphical models (GGM) are widely used to estimate network structure in domains ranging from biology to finance. In practice, data is often corrupted by latent confounders which biases inference of the underlying true graphical structure. In this paper, we compare and contrast two strategies for inference in graphical models with latent confounders: Gaussian graphical models with latent variables (LVGGM) and PCA-based removal of confounding (PCA+GGM). While these two approaches have similar goals, they are motivated by different assumptions about confounding. In this paper, we explore the connection between these two approaches and propose a new method, which combines the strengths of these two approaches. We prove the consistency and convergence rate for the PCA-based method and use these results to provide guidance about when to use each method. We demonstrate the effectiveness of our methodology using both simulations and two real-world applications.