A sparse factor model for clustering high-dimensional longitudinal data.

The elicitation of an ordinal judgment on multiple alternatives is often required in many psychological and behavioral experiments to investigate preference/choice orientation of a specific population. The Plackett-Luce model is one of the most popular and frequently applied parametric distributions to analyze rankings of a finite set of items. The present work introduces a Bayesian finite mixture of Plackett-Luce models to account for unobserved sample heterogeneity of partially ranked data. We describe an efficient way to incorporate the latent group structure in the data augmentation approach and the derivation of existing maximum likelihood procedures as special instances of the proposed Bayesian method. Inference can be conducted with the combination of the Expectation-Maximization algorithm for maximum a posteriori estimation and the Gibbs sampling iterative procedure. We additionally investigate several Bayesian criteria for selecting the optimal mixture configuration and describe diagnostic tools for assessing the fitness of ranking distributions conditionally and unconditionally on the number of ranked items. The utility of the novel Bayesian parametric Plackett-Luce mixture for characterizing sample heterogeneity is illustrated with several applications to simulated and real preference ranked data. We compare our method with the frequentist approach and a Bayesian nonparametric mixture model both assuming the Plackett-Luce model as a mixture component. Our analysis on real datasets reveals the importance of an accurate diagnostic check for an appropriate in-depth understanding of the heterogenous nature of the partial ranking data.

This dissertation focuses on the development of multiple imputation models and algorithms for high-dimensional data with variable selection structures. Leveraging on the multivariate linear mixed-effects model with missing responses for clustered data, we incorporate the variable selection routines using spike-and-slab priors within the Bayesian variable selection framework. Specific choice of these priors allow us to "force'' variables of importance (e.g. design variables or variables known to play role in missingness mechanism) into the imputation models. Our ultimate goal is to improve computational speed by removing unnecessary variables. Markov chain Monte Carlo techniques have been designed to sample from the implied posterior distributions for model unknowns as well as missing data. A computationally efficient alternative, namely, variational inference algorithms have also been developed to overcome the computational burden of Markov chain Monte Carlo for sampling the posterior predictive distribution missing data. The continuous imputation model and algorithm can be easily modified through a calibration-based rounding strategy for multiple imputation of categorical variables. We also compare two imputation strategies, i.e., the sequential imputation and the joint modeling imputation, has been made in the presence of variable selection structures. We design a joint modeling based imputation framework using a nonparametric Bayesian Dirichlet process mixture model together with a sparse precision matrix model. Two computational algorithms have been carefully designed: the Gibbs sampler and the variational Bayesian inference algorithm. The proposed methodology can be applied to both continuous and mixed continuous and categorical missing data.

We propose an approximate Bayesian approach for unsupervised feature selection and density estimation, where the importance of the features for clustering is used as the measure for feature selection. Traditional maximum-likelihood (ML) model-parameter optimization schemes estimate the feature saliencies for a fixed model structure (i.e., a fixed number of clusters). In practice, the number of clusters present in the data for mixture-based modeling is unknown. In an ML framework, the number of clusters typically needs to be ascertained prior to estimating the feature saliencies. We propose a density estimation scheme that addresses model complexity (number of clusters present) and model-parameter estimation (feature saliencies) in a single optimization framework. The approximate Bayesian approach presented here, based on the expectation propagation method, obtains a full posterior distribution on the saliency of the features, along with full posterior distribution of other model parameters (including the number of clusters) that represent the underlying statistics of the data. The performance of the algorithm, is analyzed based on its ability to identify the features salient for clustering the multivariate data.

Due to the increasing injection of intermittent power sources (solar+wind) into a common grid, dispatchable sources such as hydro power should be able to help reduce the variability in load and the variability in generation caused by the intermittent sources. A hydro generator should be able to operate short-term beyond its thermal capability limit. This requires the monitoring of internal temperatures in the hydro generator. In this paper, a thermal model of an air-cooled synchronous generator is presented, emphasizing the various aspects of parameter estimation and identifiability using Bayesian inference. Inferences are drawn from the posterior distributions of the parameters and initial conditions, dispersion (spreading) of particles and sampling efficiency, practical parameter identifiability, and model mismatch with experiments. Results show extremely narrow parameter distributions. It is early to generalize about the posterior distribution of air-related and metal-related parameters of the air-cooled synchronous generator based on the single experimental data presented here.

Markov Chain Monte Carlo (MCMC) methods have become increasingly popular for estimating the posterior probability distribution of parameters in hydrologic models. However, MCMC methods require the a priori definition of a proposal or sampling distribution, which determines the explorative capabilities and efficiency of the sampler and therefore the statistical properties of the Markov Chain and its rate of convergence. In this paper we present an MCMC sampler entitled the Shuffled Complex Evolution Metropolis algorithm (SCEM‐UA), which is well suited to infer the posterior distribution of hydrologic model parameters. The SCEM‐UA algorithm is a modified version of the original SCE‐UA global optimization algorithm developed by Duan et al. [1992]. The SCEM‐UA algorithm operates by merging the strengths of the Metropolis algorithm, controlled random search, competitive evolution, and complex shuffling in order to continuously update the proposal distribution and evolve the sampler to the posterior target distribution. Three case studies demonstrate that the adaptive capability of the SCEM‐UA algorithm significantly reduces the number of model simulations needed to infer the posterior distribution of the parameters when compared with the traditional Metropolis‐Hastings samplers.

One of the central issues in statistics and machine learning is how to select an adequate model that can automatically adapt its complexity to the observed data. In the present paper, we focus on the issue of determining the structure of clustered data, both in terms of finding the appropriate number of clusters and of modelling the right dependence structure between the observations. Bayesian nonparametric (BNP) models, which do not impose an upper limit on the number of clusters, are appropriate to avoid the required guess on the number of clusters but have been mainly developed for independent data. In contrast, Markov random fields (MRF) have been extensively used to model dependencies in a tractable manner but usually reduce to finite cluster numbers when clustering tasks are addressed. Our main contribution is to propose a general scheme to design tractable BNP-MRF priors that combine both features: no commitment to an arbitrary number of clusters and a dependence modelling. A key ingredient in this construction is the availability of a stick-breaking representation which has the threefold advantage to allowing us to extend standard discrete MRFs to infinite state space, to design a tractable estimation algorithm using variational approximation and to derive theoretical properties on the predictive distribution and the number of clusters of the proposed model. This approach is illustrated on a challenging natural image segmentation task for which it shows good performance with respect to the literature.

Mixture distributions are widely used to model data with distinct groups, providing a flexible approach to estimating density. However, Bayesian approaches for mixture models pose challenges, such as label switching in the Gibbs sampler output due to the non-identifiability of component parameters. We review advanced methods for Bayesian analysis, including the Markov chain Monte Carlo (MCMC) reversible jump algorithm and model comparison based on joint measures of fit and complexity. We also present a Bayesian regression model based on a two-component mixture model, implemented using the Gibbs sampler algorithm and applied to a dataset of time measurement differences between two clocks. Our theoretical investigation highlights the importance of latent variables in implementing the Bayesian normal mixture model with two components. When applied to the dataset, our model effectively assigned probabilities to the two states of the phenomenon under study and identified two processes with identical slopes, intercepts, and variances. Our findings demonstrate the power of Bayesian mixture models in uncovering hidden structures within complex datasets. In general, our review and application provide insight into the challenges and potential solutions for Bayesian mixture modeling and highlight the usefulness of these methods in various fields.

This paper adopts a Bayesian nonparametric mixture model where the mixing distribution belongs to the wide class of normalized homogeneous completely random measures. We propose a truncation method for the mixing distribution by discarding the weights of the unnormalized measure smaller than a threshold. We prove convergence in law of our approximation, provide some theoretical properties, and characterize its posterior distribution so that a blocked Gibbs sampler is devised. The versatility of the approximation is illustrated by two different applications. In the first the normalized Bessel random measure, encompassing the Dirichlet process, is introduced; goodness of fit indexes show its good performances as mixing measure for density estimation. The second describes how to incorporate covariates in the support of the normalized measure, leading to a linear dependent model for regression and clustering.

Network marker selection on genome-scale networks plays an important role in the understanding of biological mechanisms and disease pathologies. Recently, a Bayesian nonparametric mixture model has been developed and successfully applied for selecting genes and gene sub-networks. Hence, extending this method to a unified approach for network-based feature selection on general large-scale networks and creating an easy-to-use software package is on demand. We extended the method and developed an R package, the Bayesian network feature finder (BANFF), providing a package of posterior inference, model comparison and graphical illustration of model fitting. The model was extended to a more general form, and a parallel computing algorithm for the Markov chain Monte Carlo -based posterior inference and an expectation maximization-based algorithm for posterior approximation were added. Based on simulation studies, we demonstrate the use of BANFF on analyzing gene expression on a protein-protein interaction network. https://cran.r-project.org/web/packages/BANFF/index.html CONTACT: jiankang@umich.edu, tianwei.yu@emory.eduSupplementary information: Supplementary data are available at Bioinformatics online.

Gibbs-type random probability measures, or Gibbs-type priors, are arguably the most “natural” generalization of the celebrated Dirichlet prior. Among them the two parameter Poisson–Dirichlet prior certainly stands out in terms of mathematical tractability and interpretability of its predictive probabilities, which made it the natural candidate in a plethora of applications. Given a random sample of size n from an arbitrary Gibbs-type prior, we show that the corresponding predictive probabilities admit a large n approximation, with an error term vanishing as o(1/n), which maintains the same desirable features as the predictive probabilities of the two parameter Poisson–Dirichlet prior. Our result is illustrated through an extensive simulation study, which includes an application in the context of Bayesian nonparametric mixture modeling.

Tractable Bayesian density regression via logit stick-breaking priors

Recommender systems have become an important area of research with numerous applications on e-commerce. This chapter introduces a joint statistical model for user preferences and item features that can serve as the basis for a recommendation about recently published scientific papers. The model is constructed using ideas from the literature on Bayesian nonparametric mixture modeling. More specifically, user preferences are modeled using an Infinite Relational Model (IRM) in which both users and items are independently partitioned into homogeneous groups, while item features are modeled using a topic model, which also partitions items into groups with homogenous features. Information is shared across both components of the model through a common partition of items. Hence, the model is a hybrid system that combines ideas from collaborative and content-based filtering. The chapter discusses three different computational strategies, including a Markov chain Monte Carlo algorithm for full posterior inference, an iterated conditional maximization algorithm, and a mean-field variational algorithm for point estimation and prediction in large datasets where Markov chain Monte Carlo approaches might not be practical. The model is illustrated through simulation studies and by analyzing data from CiteULike.

Superpixels provide a useful intermediate image representation. Existing superpixel methods, however, suffer from at least some of the following drawbacks: 1) topology is handled heuristically; 2) the number of superpixels is either predefined or estimated at a prohibitive cost; 3) lack of adaptiveness. As a remedy, we propose a novel probabilistic model, self-coined Bayesian Adaptive Superpixel Segmentation (BASS), together with an efficient inference. BASS is a Bayesian nonparametric mixture model that also respects topology and favors spatial coherence. The optimizationbased and topology-aware inference is parallelizable and implemented in GPU. Quantitatively, BASS achieves results that are either better than the state-of-the-art or close to it, depending on the performance index and/or dataset. Qualitatively, we argue it achieves the best results; we demonstrate this by not only subjective visual inspection but also objective quantitative performance evaluation of the downstream application of face detection. Our code is available at https://github.com/uzielroy/BASS.

We investigate the use of a large class of discrete random probability measures, which is referred to as the class $$\mathcal {Q}$$Q, in the context of Bayesian nonparametric mixture modeling. The class $$\mathcal {Q}$$Q encompasses both the the two-parameter Poisson---Dirichlet process and the normalized generalized Gamma process, thus allowing us to comparatively study the inferential advantages of these two well-known nonparametric priors. Apart from a highly flexible parameterization, the distinguishing feature of the class $$\mathcal {Q}$$Q is the availability of a tractable posterior distribution. This feature, in turn, leads to derive an efficient marginal MCMC algorithm for posterior sampling within the framework of mixture models. We demonstrate the efficacy of our modeling framework on both one-dimensional and multi-dimensional datasets.

In recent years there has been great interest in making inference for gene expression data collected over time. In this article, we describe a Bayesian hierarchical mixture model for partitioning such data. While conventional approaches cluster the observed data, we assume a nonparametric, random walk model, and partition on the basis of the parameters of this model. The model is flexible and can be tuned to the specific context, respects the order of observations within each curve, acknowledges measurement error, and allows prior knowledge on parameters to be incorporated. The number of partitions may also be treated as unknown, and inferred from the data, in which case computation is carried out via a birth-death Markov chain Monte Carlo algorithm. We first examine the behavior of the model on simulated data, along with a comparison with more conventional approaches, and then analyze meiotic expression data collected over time on fission yeast genes.