Loss function, unbiasedness, and optimality of Gaussian graphical model selection

Graphical models provide a framework for exploration of multivariate dependence patterns. The connection between graph and statistical model is made by identifying the vertices of the graph with the observed variables and translating the pattern of edges in the graph into a pattern of conditional independences that is imposed on the variables’ joint distribution. Focusing on Gaussian models, we review classical graphical models. For these models the defining conditional independences are equivalent to vanishing of certain (partial) correlation coefficients associated with individual edges that are absent from the graph. Hence, Gaussian graphical model selection can be performed by multiple testing of hypotheses about vanishing (partial) correlation coefficients. We show and exemplify how this approach allows one to perform model selection while controlling error rates for incorrect edge inclusion.

This paper introduces a MATLAB toolbox, MGraph, which applies graphical models as a natural environment to formulate and solve problems in microarray data analysis. MGraph with its graphical interface allows the user to predict genetic regulatory networks by a graphical gaussian model (GGM), and to quantify the effects of different experimental treatment conditions on gene expression profiles by a graphical log-linear model (GLM). The power of graphical models was explored and illustrated through two example applications. First, four MAPK pathways in yeast were meaningfully reconstructed through GGM. Second, GLM was used to quantify the contributions of sex, genotype and age to transcriptional variance in Drosophila melanogaster. This application may provide a valuable aid in the prediction of genetic regulatory networks, as well as in investigations of various experimental conditions that affect global gene expression profiles. The MATLAB program MGraph is freely available at http://www.uio.no/~junbaiw/mgraph/mgraph.html for academics.

Gaussian Graphical models (GGM) are widely used to estimate network structure in domains ranging from biology to finance. In practice, data is often corrupted by latent confounders which biases inference of the underlying true graphical structure. In this paper, we compare and contrast two strategies for inference in graphical models with latent confounders: Gaussian graphical models with latent variables (LVGGM) and PCA-based removal of confounding (PCA+GGM). While these two approaches have similar goals, they are motivated by different assumptions about confounding. In this paper, we explore the connection between these two approaches and propose a new method, which combines the strengths of these two approaches. We prove the consistency and convergence rate for the PCA-based method and use these results to provide guidance about when to use each method. We demonstrate the effectiveness of our methodology using both simulations and two real-world applications.

It is well known that regression methods designed for clean data will lead to erroneous results if directly applied to corrupted data. Despite the recent methodological and algorithmic advances in Gaussian graphical model estimation, how to achieve efficient and scalable estimation under contaminated covariates is unclear. Here a new methodology called convex conditioned innovative scalable efficient estimation (COCOISEE) for Gaussian graphical model under both additive and multiplicative measurement errors is developed. It combines the strengths of the innovative scalable efficient estimation in Gaussian graphical model and the nearest positive semi-definite matrix projection, thus enjoying stepwise convexity and scalability. Comprehensive theoretical guarantees are provided and the effectiveness of the proposed methodology is demonstrated through numerical studies.

This doctoral thesis contributes to the advancement of statistical analyses of gene expression profiles using Gaussian graphical models (GGMs). The main focus has been on studying the properties of GGMs obtained with the LW shrinkage. The premise is that, despite its advantages, the shrinkage approach introduces some biases that have not been studied in sufficient detail. These biases have the potential of obscuring the interpretation of the network structure and impeding the validation of earlier analyses. In this sense, this thesis may be considered a continuation of the works of Schäfer and Strimmer, where the LW-shrinkage was originally used to model GRNs with GGMs. The thesis is organized as follows; Chapter 1 presents a test of statistical significance for GGMs based on the LW-shrinkage. Here the probability density of the ‘shrunk’ partial correlation is derived by means of geometric arguments. In Chapter 2 a network analysis of (matched) nasal and bronchial expression profiles is presented. The method developed in the previous chapter is employed to explore whether expression profiles from nasal epithelial cells can be used as a proxy for bronchial epithelial cells. Chapter 3 shows the existence of a non-linear bias on the partial correlations obtained with the LW-shrinkage. This bias is removed via ‘un-shrinking’; a new concept that de-regularizes the partial correlation. In Chapter 4 the LW-shrinkage is revisited from a data-level perspective. The goal is to explore the correspondence between the shrinkage-based covariance matrix and the dataset.

Nitrogen is an essential nutrient for all life forms. Like most unicellular organisms, the yeast Saccharomyces cerevisiae transports and catabolizes good nitrogen sources in preference to poor ones. Nitrogen catabolite repression (NCR) refers to this selection mechanism. We propose an approach based on Gaussian graphical models (GGMs), which enable to distinguish direct from indirect interactions between genes, to identify putative NCR genes from putative NCR regulatory motifs and over-represented motifs in the upstream noncoding sequences of annotated NCR genes. Because of the high-dimensionality of the data, we use a shrinkage estimator of the covariance matrix to infer the GGMs. We show that our approach makes significant and biologically valid predictions. We also show that GGMs are more effective than models that rely on measures of direct interactions between genes.

The multivariate adaptive regression spline (MARS) is one of the alternative nonparametric approaches of the Gaussian graphical model (GGM) in order to construct the undirected networks of complex biological systems. In GGM, it is assumed that the binary form of the estimated precision matrix, computed from the systems' elements under their conditional independencies, represents the undirected interactions of these elements by using the lasso regression. From previous studies, it has been shown that if MARS is estimated by discarding all interactions and nonlinear components, resulting in a sole inference of the main effects, it can be a strong alternative of GGM. We call this type of the summarized MARS model the lasso-based MARS model. Furthermore, in the construction of the networks via the optimal MARS model, it is assumed that the estimated regression coefficients, i.e., the important terms of the estimated model, can be the indication of the pairwise links between the species of the systems. Hereby, in this study, we investigate whether the inference of the networks via the Pearson, Spearman, and Kendall's tau correlation coefficients has better performance than the networks constructed by the important regression coefficients in the lasso-based MARS model. In the analyses, we use both simulated and real datasets and compare the estimated networks by various accuracy measures from specificity and F-measure to Matthews correlation coefficient. From the analyses, we find that the suggested approaches improve all the listed criteria significantly without any computational cost.

A major challenge in contemporary neuroscience is to analyze data from large numbers of neurons recorded simultaneously across many experimental replications (trials), where the data are counts of neural firing events, and one of the basic problems is to characterize the dependence structure among such multivariate counts. Methods of estimating high-dimensional covariation based on ℓ 1-regularization are most appropriate when there are a small number of relatively large partial correlations, but in neural data there are often large numbers of relatively small partial correlations. Furthermore, the variation across trials is often confounded by Poisson-like variation within trials. To overcome these problems we introduce a comprehensive methodology that imbeds a Gaussian graphical model into a hierarchical structure: the counts are assumed Poisson, conditionally on latent variables that follow a Gaussian graphical model, and the graphical model parameters, in turn, are assumed to depend on physiologically-motivated covariates, which can greatly improve correct detection of interactions (non-zero partial correlations). We develop a Bayesian approach to fitting this covariate-adjusted generalized graphical model and we demonstrate its success in simulation studies. We then apply it to data from an experiment on visual attention, where we assess functional interactions between neurons recorded from two brain areas.

This paper concerns the specification, and performance, of scale-free prior distributions with a view toward large-scale network inference from small-sample data sets. We devise three scale-free priors and implement them in the framework of Gaussian graphical models. Gaussian graphical models are used in gene network inference where high-throughput data describing a large number of variables with comparatively few samples are frequently analyzed by practitioners. And, although there is a consensus that many such networks are scale-free, the modus operandi is to assign a random network prior. Simulations demonstrate that the scale-free priors outperform the random network prior at recovering scale-free trees with degree exponents near 2, such as are characteristic of many real-world systems. On the other hand, the random network prior compares favorably at recovering scale-free trees characterized by larger degree exponents.

Biological networks provide additional information for the analysis of human diseases, beyond the traditional analysis that focuses on single variables. Gaussian graphical model (GGM), a probability model that characterizes the conditional dependence structure of a set of random variables by a graph, has wide applications in the analysis of biological networks, such as inferring interaction or comparing differential networks. However, existing approaches are either not statistically rigorous or are inefficient for high-dimensional data that include tens of thousands of variables for making inference. In this study, we propose an efficient algorithm to implement the estimation of GGM and obtain p-value and confidence interval for each edge in the graph, based on a recent proposal by Ren et al., 2015. Through simulation studies, we demonstrate that the algorithm is faster by several orders of magnitude than the current implemented algorithm for Ren et al. without losing any accuracy. Then, we apply our algorithm to two real data sets: transcriptomic data from a study of childhood asthma and proteomic data from a study of Alzheimer’s disease. We estimate the global gene or protein interaction networks for the disease and healthy samples. The resulting networks reveal interesting interactions and the differential networks between cases and controls show functional relevance to the diseases. In conclusion, we provide a computationally fast algorithm to implement a statistically sound procedure for constructing Gaussian graphical model and making inference with high-dimensional biological data. The algorithm has been implemented in an R package named “FastGGM”.

Constructing expression networks using transcriptomic data is an effective approach for studying gene regulation. A popular approach for constructing such a network is based on the Gaussian graphical model (GGM), in which an edge between a pair of genes indicates that the expression levels of these two genes are conditionally dependent, given the expression levels of all other genes. However, GGMs are not appropriate for non-Gaussian data, such as those generated in RNA-seq experiments. We propose a novel statistical framework that maximizes a penalized likelihood, in which the observed count data follow a Poisson log-normal distribution. To overcome the computational challenges, we use Laplace's method to approximate the likelihood and its gradients, and apply the alternating directions method of multipliers to find the penalized maximum likelihood estimates. The proposed method is evaluated and compared with GGMs using both simulated and real RNA-seq data. The proposed method shows improved performance in detecting edges that represent covarying pairs of genes, particularly for edges connecting low-abundant genes and edges around regulatory hubs.

Clustering genes with high correlations will group genes with close expression profiles, defining clusters of co-expressed genes. However, such correlations do not provide any clue on the chain of information going from gene to gene. Partial correlation consists in quantifying the correlation between two genes after excluding the effects of the other genes. Partial correlation thus makes it possible to distinguish between the correlation of two genes due to direct causal relationships from the correlation that originates via intermediate genes. In this chapter, Gaussian graphical model (GGM) learning is set up as a covariate selection problem. Two least absolute shrinkage and selection operator (LASSO)-type techniques are described, the graphical LASSO approach and the neighborhood selection. Then two extensions to the classical GGM are presented. GGMs are extended in structured GGMs, to account for modularity, and more generally heterogeneity in the gene connection features. The extension using a biological prior on the network structure is illustrated on real data.

BackgroundGraphical Gaussian models are popular tools for the estimation of (undirected) gene association networks from microarray data. A key issue when the number of variables greatly exceeds the number of samples is the estimation of the matrix of partial correlations. Since the (Moore-Penrose) inverse of the sample covariance matrix leads to poor estimates in this scenario, standard methods are inappropriate and adequate regularization techniques are needed. Popular approaches include biased estimates of the covariance matrix and high-dimensional regression schemes, such as the Lasso and Partial Least Squares.ResultsIn this article, we investigate a general framework for combining regularized regression methods with the estimation of Graphical Gaussian models. This framework includes various existing methods as well as two new approaches based on ridge regression and adaptive lasso, respectively. These methods are extensively compared both qualitatively and quantitatively within a simulation study and through an application to six diverse real data sets. In addition, all proposed algorithms are implemented in the R package "parcor", available from the R repository CRAN.ConclusionIn our simulation studies, the investigated non-sparse regression methods, i.e. Ridge Regression and Partial Least Squares, exhibit rather conservative behavior when combined with (local) false discovery rate multiple testing in order to decide whether or not an edge is present in the network. For networks with higher densities, the difference in performance of the methods decreases. For sparse networks, we confirm the Lasso's well known tendency towards selecting too many edges, whereas the two-stage adaptive Lasso is an interesting alternative that provides sparser solutions. In our simulations, both sparse and non-sparse methods are able to reconstruct networks with cluster structures. On six real data sets, we also clearly distinguish the results obtained using the non-sparse methods and those obtained using the sparse methods where specification of the regularization parameter automatically means model selection. In five out of six data sets, Partial Least Squares selects very dense networks. Furthermore, for data that violate the assumption of uncorrelated observations (due to replications), the Lasso and the adaptive Lasso yield very complex structures, indicating that they might not be suited under these conditions. The shrinkage approach is more stable than the regression based approaches when using subsampling.

Background Construction of networks from cross-sectional biological data is increasingly common. Many recent methods have been based on Gaussian graphical modeling, and prioritize estimation of conditional pairwise dependencies among nodes in the network. However, challenges remain on how specific paths through the resultant network contribute to overall ‘network-level’ correlations. For biological applications, understanding these relationships is particularly relevant for parsing structural information contained in complex subnetworks.ResultsWe propose the pair-path subscore (PPS), a method for interpreting Gaussian graphical models at the level of individual network paths. The scoring is based on the relative importance of such paths in determining the Pearson correlation between their terminal nodes. PPS is validated using human metabolomics data from the Hyperglycemia and adverse pregnancy outcome (HAPO) study, with observations confirming well-documented biological relationships among the metabolites. We also highlight how the PPS can be used in an exploratory fashion to generate new biological hypotheses. Our method is implemented in the R package pps, available at https://github.com/nathan-gill/pps.ConclusionsThe PPS can be used to probe network structure on a finer scale by investigating which paths in a potentially intricate topology contribute most substantially to marginal behavior. Adding PPS to the network analysis toolkit may enable researchers to ask new questions about the relationships among nodes in network data.

Tests for Gaussian graphical models