Sparse Bayesian infinite factor models

It is well recognized that the effects of drugs are far beyond targeting individual proteins, but rather influencing the complex interactions among many relevant biological pathways. Genome-wide expression profiling before and after drug treatment has become a powerful approach for capturing a global snapshot of cellular response to drugs, as well as to understand drugs' mechanism of action. Therefore, it is of great interest to analyze this type of transcriptomic profiling data for the identification of pathways responsive to different drugs. However, few computational tools exist for this task. We have developed FacPad, a Bayesian sparse factor model, for the inference of pathways responsive to drug treatments. This model represents biological pathways as latent factors and aims to describe the variation among drug-induced gene expression alternations in terms of a much smaller number of latent factors. We applied this model to the Connectivity Map data set (build 02) and demonstrated that FacPad is able to identify many drug-pathway associations, some of which have been validated in the literature. Although this method was originally designed for the analysis of drug-induced transcriptional alternation data, it can be naturally applied to many other settings beyond polypharmacology. The R package 'FacPad' is publically available at: http://cran.open-source-solution.org/web/packages/FacPad/.

Factors models are commonly used to analyze high-dimensional data in both single-study and multi-study settings. Bayesian inference for such models relies on Markov chain Monte Carlo (MCMC) methods, which scale poorly as the number of studies, observations, or measured variables increase. To address this issue, we propose new variational inference algorithms to approximate the posterior distribution of Bayesian latent factor models using the multiplicative gamma process shrinkage prior. The proposed algorithms provide fast approximate inference at a fraction of the time and memory of MCMC-based implementations while maintaining comparable accuracy in characterizing the data covariance matrix. We conduct extensive simulations to evaluate our proposed algorithms and show their utility in estimating the model for high-dimensional multi-study gene expression data in ovarian cancers. Overall, our proposed approaches enable more efficient and scalable inference for factor models, facilitating their use in high-dimensional settings. An R package VIMSFA implementing our methods is available on GitHub (github.com/blhansen/VI-MSFA). Supplementary materials for this article are available online.

The development of simulation-based methods, such as Markov chain Monte Carlo (MCMC), has contributed to an increased interest in the Bayesian framework as an alternative to deal with factor models. Many studies have used Bayesian factor analysis to explore gene expression data. We are particularly interested in the application of a sparse latent factor model (SLFM) based on sparsity priors (mixtures) to assess the significance of factors. The SLFM measures how strong the observed coherent expression pattern is in the data, which is an important source of information to evaluate gene activity. In the literature, this type of model has shown better results than other approaches intended for identification of patterns and metagene groups related to the underlying biology. However, a full Bayesian factor model relying on MCMC algorithms has an expensive computational cost, which makes it unattractive for general users. In this paper, we present the package slfm which uses C++ implementation via Rcpp to improve the computational performance of the SLFM within the widely used statistical tool R. We investigate real and simulated microarray data related to breast cancer.

Covariance estimation for multiple groups is a key feature for drawing inference from a heterogeneous population. One should seek to share information about common features in the dependence structures across the various groups. In this paper, we introduce a novel approach for estimating the covariance matrices for multiple groups using a hierarchical latent factor model that shrinks the factor loadings across groups toward a global value. Using a sparse spike and slab model on these loading coefficients allows for a sparse formulation of our model. Parameter estimation is accomplished through a Markov chain Monte Carlo scheme, and a model selection approach is used to select the number of factors to use. We validate our model through extensive simulation studies. Finally, we apply our methodology to the NICHD Consecutive Pregnancies Study to estimate the correlations between birth weights and gestational ages of three consecutive birth within four different subgroups (underweight, normal, overweight, and obese) of women.

Patient reported outcomes are gaining more attention in patient-centered health outcomes research and quality of life studies as important indicators of clinical outcomes, especially for patients with chronic diseases. Factor analysis is ideal for measuring patient reported outcomes. If there is heterogeneity in the patient population and when sample size is small, differential item functioning and convergence issues are challenges for applying factor models. Bayesian hierarchical factor analysis can assess health disparity by assessing for di˙erential item functioning, while avoiding convergence problems. We conducted a simulation study and used an empirical example with American Indian minorities to show thatffitting a Bayesian hierarchical factor model is an optimal solution regardless of heterogeneity of population and sample size.

This work introduces a novel estimation method, called LOVE, of the entries and structure of a loading matrix $A$ in a latent factor model $X=AZ+E$, for an observable random vector $X\\in \\mathbb{R}^{p}$, with correlated unobservable factors $Z\\in \\mathbb{R}^{K}$, with $K$ unknown, and uncorrelated noise $E$. Each row of $A$ is scaled, and allowed to be sparse. In order to identify the loading matrix $A$, we require the existence of pure variables, which are components of $X$ that are associated, via $A$, with one and only one latent factor. Despite the fact that the number of factors $K$, the number of the pure variables and their location are all unknown, we only require a mild condition on the covariance matrix of $Z$, and a minimum of only two pure variables per latent factor to show that $A$ is uniquely defined, up to signed permutations. Our proofs for model identifiability are constructive, and lead to our novel estimation method of the number of factors and of the set of pure variables, from a sample of size $n$ of observations on $X$. This is the first step of our LOVE algorithm, which is optimization-free, and has low computational complexity of order $p^{2}$. The second step of LOVE is an easily implementable linear program that estimates $A$. We prove that the resulting estimator is near minimax rate optimal for $A$, with respect to the $\\|\\ \\|_{\\infty ,q}$ loss, for $q\\geq 1$, up to logarithmic factors in $p$, and that it can be minimax-rate optimal in many cases of interest.\n¶ The model structure is motivated by the problem of overlapping variable clustering, ubiquitous in data science. We define the population level clusters as groups of those components of $X$ that are associated, via the matrix $A$, with the same unobservable latent factor, and multifactor association is allowed. Clusters are respectively anchored by the pure variables, and form overlapping subgroups of the $p$-dimensional random vector $X$. The Latent model approach to OVErlapping clustering is reflected in the name of our algorithm, LOVE.\n¶ The third step of LOVE estimates the clusters from the support of the columns of the estimated $A$. We guarantee cluster recovery with zero false positive proportion, and with false negative proportion control. The practical relevance of LOVE is illustrated through the analysis of a RNA-seq data set, devoted to determining the functional annotation of genes with unknown function.

This article introduces novel and practicable Bayesian factor analysis frameworks that are computationally feasible for moderate to large spatiotemporal data. Previous Bayesian analysis of spatiotemporal data has used a Bayesian factor model with separable temporal latent factors and spatial factor loadings, along with stick-breaking process priors on the loadings to enable clustering of spatial locations. Such a flexible Bayesian model, however, faces a prohibitively high computational cost in posterior sampling when the spatial and temporal dimensions increase to a couple hundred. We address this computational challenge with several speed-up proposals. We integrate a new slice sampling algorithm that permits varying numbers of spatial mixture components across all latent factors and guarantees them to be nonincreasing through the posterior sampling iterations, thus, effectively reducing the number of mixture parameters. Additionally, we introduce a spatial latent nearest-neighbor Gaussian process prior and new sequential updating algorithms for the spatially varying latent variables in the stick-breaking process prior. Our new models and sampling algorithms exhibit significantly enhanced computational scalability and storage efficiency and possess powerful inferential capabilities for both spatiotemporal prediction and clustering of spatial locations with similar temporal trajectories. The improvement in computational efficiency and inferential performance is substantiated by extensive simulation experiments.

The authors present a Bayesian simultaneous choice factor model that measures consumers' willingness to pay for brand-image associations. Previous research has found that general brand effects influence a brand's scores on specific image dimensions. To investigate the value of general versus specific brand image, the authors specify a higher-order factor model in which a set of correlated factor scores arise from a general brand factor and a set of orthogonal residual scores that measure the specific dimensions of brand image. The general brand factor is consistent with the concept of a halo effect, which theory ascribes to either an overall evaluative effect or errors in cognition. The authors apply the model to stated preference data on branded midsized sedans accompanied by data on consumer brand-image associations. The authors find that there is substantial value for the specific dimensions of brand image, but only after controlling for the general brand effect with the higher-order factor decomposition.

Sparse Bayesian factor models are routinely implemented for parsimonious dependence modeling and dimensionality reduction in high-dimensional applications. We provide theoretical understanding of such Bayesian procedures in terms of posterior convergence rates in inferring high-dimensional covariance matrices where the dimension can be larger than the sample size. Under relevant sparsity assumptions on the true covariance matrix, we show that commonly-used point mass mixture priors on the factor loadings lead to consistent estimation in the operator norm even when $p\gg n$. One of our major contributions is to develop a new class of continuous shrinkage priors and provide insights into their concentration around sparse vectors. Using such priors for the factor loadings, we obtain similar rate of convergence as obtained with point mass mixture priors. To obtain the convergence rates, we construct test functions to separate points in the space of high-dimensional covariance matrices using insights from random matrix theory; the tools developed may be of independent interest. We also derive minimax rates and show that the Bayesian posterior rates of convergence coincide with the minimax rates upto a $\sqrt{\log n}$ term.

Bayesian factor analysis with uncertain functional constraints about factor loadings

This dissertation focuses on the development of multiple imputation models and algorithms for high-dimensional data with variable selection structures. Leveraging on the multivariate linear mixed-effects model with missing responses for clustered data, we incorporate the variable selection routines using spike-and-slab priors within the Bayesian variable selection framework. Specific choice of these priors allow us to "force'' variables of importance (e.g. design variables or variables known to play role in missingness mechanism) into the imputation models. Our ultimate goal is to improve computational speed by removing unnecessary variables. Markov chain Monte Carlo techniques have been designed to sample from the implied posterior distributions for model unknowns as well as missing data. A computationally efficient alternative, namely, variational inference algorithms have also been developed to overcome the computational burden of Markov chain Monte Carlo for sampling the posterior predictive distribution missing data. The continuous imputation model and algorithm can be easily modified through a calibration-based rounding strategy for multiple imputation of categorical variables. We also compare two imputation strategies, i.e., the sequential imputation and the joint modeling imputation, has been made in the presence of variable selection structures. We design a joint modeling based imputation framework using a nonparametric Bayesian Dirichlet process mixture model together with a sparse precision matrix model. Two computational algorithms have been carefully designed: the Gibbs sampler and the variational Bayesian inference algorithm. The proposed methodology can be applied to both continuous and mixed continuous and categorical missing data.

BackgroundIt has been shown that gene expression in human tissues is heritable, thus predicting gene expression using only SNPs becomes possible. The prediction of gene expression can offer important implications on the genetic architecture of individual functional associated SNPs and further interpretations of the molecular basis underlying human diseases.MethodsWe compared three types of methods for predicting gene expression using only cis-SNPs, including the polygenic model, i.e. linear mixed model (LMM), two sparse models, i.e. Lasso and elastic net (ENET), and the hybrid of LMM and sparse model, i.e. Bayesian sparse linear mixed model (BSLMM). The three kinds of prediction methods have very different assumptions of underlying genetic architectures. These methods were evaluated using simulations under various scenarios, and were applied to the Geuvadis gene expression data.ResultsThe simulations showed that these four prediction methods (i.e. Lasso, ENET, LMM and BSLMM) behaved best when their respective modeling assumptions were satisfied, but BSLMM had a robust performance across a range of scenarios. According to R2 of these models in the Geuvadis data, the four methods performed quite similarly. We did not observe any clustering or enrichment of predictive genes (defined as genes with R2 ≥ 0.05) across the chromosomes, and also did not see there was any clear relationship between the proportion of the predictive genes and the proportion of genes in each chromosome. However, an interesting finding in the Geuvadis data was that highly predictive genes (e.g. R2 ≥ 0.30) may have sparse genetic architectures since Lasso, ENET and BSLMM outperformed LMM for these genes; and this observation was validated in another gene expression data. We further showed that the predictive genes were enriched in approximately independent LD blocks.ConclusionsGene expression can be predicted with only cis-SNPs using well-developed prediction models and these predictive genes were enriched in some approximately independent LD blocks. The prediction of gene expression can shed some light on the functional interpretation for identified SNPs in GWASs.

Bayesian Latent Factor Model for Multi-target Inference

In event-based social networks, such as Meetup, social groups refer to self-organized communities that consist of users who share the same interests. In many real-world scenarios, users usually have social group preference and join interested social groups to attend events. It is therefore necessary to consider the influence of social groups to improve the event recommendation performance; however, existing event recommendation models generally consider users' individual preferences and neglect the influence of social groups. To this end, we propose a new Bayesian latent factor model SogBmf that combines social group influence and individual preference for event recommendation. Experiments on real-world data sets demonstrate the effectiveness of the proposed method.

We develop a Bayesian latent factor model of the joint long-run evolution of GDP per capita for 113 countries over the 118 years from 1900 to 2017. We find considerable heterogeneity in rates of convergence, including rates for some countries that are so slow that they might not converge (or diverge) in century-long samples, and a sparse correlation pattern (“convergence clubs”) between countries. The joint Bayesian structure allows us to compute a joint predictive distribution for the output paths of these countries over the next 100 years. This predictive distribution can be used for simulations requiring projections into the deep future, such as estimating the costs of climate change. The model's pooling of information across countries results in tighter prediction intervals than are achieved using univariate information sets. Still, even using more than a century of data on many countries, the 100-year growth paths exhibit very wide uncertainty.