Dirichlet process mixture models using matrix‐generalized half‐t distribution

The aim of this study was to implement a Dirichlet process mixture (DPM) model for automatic tumor edge identification on (18)F-fluorodeoxyglucose positron emission tomography ((18)F-FDG PET) images by optimizing the parameters on which the algorithm depends, to validate it experimentally, and to test its robustness. The DPM model belongs to the class of the Bayesian nonparametric models and uses the Dirichlet process prior for flexible nonparametric mixture modeling, without any preliminary choice of the number of mixture components. The DPM algorithm implemented in the statistical software package R was used in this work. The contouring accuracy was evaluated on several image data sets: on an IEC phantom (spherical inserts with diameter in the range 10-37 mm) acquired by a Philips Gemini Big Bore PET-CT scanner, using 9 different target-to-background ratios (TBRs) from 2.5 to 70; on a digital phantom simulating spherical/uniform lesions and tumors, irregular in shape and activity; and on 20 clinical cases (10 lung and 10 esophageal cancer patients). The influence of the DPM parameters on contour generation was studied in two steps. In the first one, only the IEC spheres having diameters of 22 and 37 mm and a sphere of the digital phantom (41.6 mm diameter) were studied by varying the main parameters until the diameter of the spheres was obtained within 0.2% of the true value. In the second step, the results obtained for this training set were applied to the entire data set to determine DPM based volumes of all available lesions. These volumes were compared to those obtained by applying already known algorithms (Gaussian mixture model and gradient-based) and to true values, when available. Only one parameter was found able to significantly influence segmentation accuracy (ANOVA test). This parameter was linearly connected to the uptake variance of the tested region of interest (ROI). In the first step of the study, a calibration curve was determined to automatically generate the optimal parameter from the variance of the ROI. This "calibration curve" was then applied to contour the whole data set. The accuracy (mean discrepancy between DPM model-based contours and reference contours) of volume estimation was below (1 ± 7)% on the whole data set (1 SD). The overlap between true and automatically segmented contours, measured by the Dice similarity coefficient, was 0.93 with a SD of 0.03. The proposed DPM model was able to accurately reproduce known volumes of FDG concentration, with high overlap between segmented and true volumes. For all the analyzed inserts of the IEC phantom, the algorithm proved to be robust to variations in radius and in TBR. The main advantage of this algorithm was that no setting of DPM parameters was required in advance, since the proper setting of the only parameter that could significantly influence the segmentation results was automatically related to the uptake variance of the chosen ROI. Furthermore, the algorithm did not need any preliminary choice of the optimum number of classes to describe the ROIs within PET images and no assumption about the shape of the lesion and the uptake heterogeneity of the tracer was required.

In machine learning and data mining tasks, Clustering is considered to be one of the most important techniques. The same sorts of documents are grouped by performing clustering techniques. Similarity measuring is used to determine transaction relationships. Hierarchical clustering model generates tree structured results. Partitioned based clustering produces the result in grid format. Text documents are unstructured data values with high dimensional attributes. Document clustering group transforms unlabeled text documents into meaningful clusters. In the event of document grouping process, traditional clustering methods require cluster count (K). Clustering accuracy degrades drastically with reference to the unsuitable cluster count. It is observed that document features are automatically partitioned into two groups namely — discriminative words and non-discriminative words. In particular, discriminative words are only useful for grouping documents. The involvement of non-discriminative words confuses the clustering process and leads to poor clustering solution in return. A variation inference algorithm is used to infer the document collection structure and partition of document words simultaneously. Dirichlet Process Mixture (DPM) model is used to partition documents in a way utilizing both the data likelihood and the clustering property of the Dirichlet Process (DP). Dirichlet Process Mixture Model for Feature Extraction (DPMFE) is used to discover the latent cluster structure based on the DPM model and it is performed without involving the number of clusters as input.

This paper presents an empirical study of endogenous treatment effects in the presence of heterogeneous responses. We estimate the incentive and selection effects of having prescription drug coverage on total drug expenditures using a sample from the Medicare Current Beneficiary Survey (MCBS). A Dirichlet process mixture (DPM) model is used to model the heterogeneity in treatment effects. Rather than estimating a finite mixture model with a fixed number of components, we specify a DP prior on the parameters, thus allowing the data and prior knowledge to determine the number of components. Endogeneity is modeled by the covariance of the error terms of the selection and the outcome equations in a two-equation selection model. We find that there are strong incentive and advantageous selection effects, with the average treatment effect and the average treatment effect for the treated estimated at $1132 and $858, respectively. Substantial heterogeneity is found to exist in the selection effects, particularly for those having lower drug expenditures. Copyright © 2014 John Wiley & Sons, Ltd.

Unsupervised outlier detection using random subspace and subsampling ensembles of Dirichlet process mixtures

In the Bayesian mixture modeling framework it is possible to infer the necessary number of components to model the data and therefore it is unnecessary to explicitly restrict the number of components. Nonparametric mixture models sidestep the problem of finding the "correct" number of mixture components by assuming infinitely many components. In this paper Dirichlet process mixture (DPM) models are cast as infinite mixture models and inference using Markov chain Monte Carlo is described. The specification of the priors on the model parameters is often guided by mathematical and practical convenience. The primary goal of this paper is to compare the choice of conjugate and non-conjugate base distributions on a particular class of DPM models which is widely used in applications, the Dirichlet process Gaussian mixture model (DPGMM). We compare computational efficiency and modeling performance of DPGMM defined using a conjugate and a conditionally conjugate base distribution. We show that better density models can result from using a wider class of priors with no or only a modest increase in computational effort.

Hidden Markov random field (HMRF) models are widely used for image segmentation, as they appear naturally in problems where a spatially constrained clustering scheme is asked for. A major limitation of HMRF models concerns the automatic selection of the proper number of their states, i.e., the number of region clusters derived by the image segmentation procedure. Existing methods, including likelihood- or entropy-based criteria, and reversible Markov chain Monte Carlo methods, usually tend to yield noisy model size estimates while imposing heavy computational requirements. Recently, Dirichlet process (DP, infinite) mixture models have emerged in the cornerstone of nonparametric Bayesian statistics as promising candidates for clustering applications where the number of clusters is unknown a priori; infinite mixture models based on the original DP or spatially constrained variants of it have been applied in unsupervised image segmentation applications showing promising results. Under this motivation, to resolve the aforementioned issues of HMRF models, in this paper, we introduce a nonparametric Bayesian formulation for the HMRF model, the infinite HMRF model, formulated on the basis of a joint Dirichlet process mixture (DPM) and Markov random field (MRF) construction. We derive an efficient variational Bayesian inference algorithm for the proposed model, and we experimentally demonstrate its advantages over competing methodologies.

ABSTRACTA natural Bayesian approach for mixture models with an unknown number of components is to take the usual finite mixture model with symmetric Dirichlet weights, and put a prior on the number of components—that is, to use a mixture of finite mixtures (MFM). The most commonly used method of inference for MFMs is reversible jump Markov chain Monte Carlo, but it can be nontrivial to design good reversible jump moves, especially in high-dimensional spaces. Meanwhile, there are samplers for Dirichlet process mixture (DPM) models that are relatively simple and are easily adapted to new applications. It turns out that, in fact, many of the essential properties of DPMs are also exhibited by MFMs—an exchangeable partition distribution, restaurant process, random measure representation, and stick-breaking representation—and crucially, the MFM analogues are simple enough that they can be used much like the corresponding DPM properties. Consequently, many of the powerful methods developed for inference in DPMs can be directly applied to MFMs as well; this simplifies the implementation of MFMs and can substantially improve mixing. We illustrate with real and simulated data, including high-dimensional gene expression data used to discriminate cancer subtypes. Supplementary materials for this article are available online.

The accurate modeling of uncertain variables, such as wind power and load uncertainty, is an essential foundation for the planning operation and scheduling decisions in power systems. Currently, Gaussian mixture model (GMM) and Dirichlet process mixture model (DPMM) are the primary techniques used to characterize uncertainties in power systems, which are commonly solved by expectation-maximization (EM) algorithm. However, for the massive data of uncertain variables, the algorithm encounters challenges in accurately obtaining GMM and DPMM with a lower time consumption. To address this issue, we propose a method for GMM uncertainty modeling in power systems considering the mutual assistance of latent variables. Specifically, the GMM of uncertain variables is first constructed, and the conditional probability is employed to characterize the mutual assistance of latent variables. Then, an improved EM algorithm is developed to obtain the optimal GMM parameters, in which the expectation step (E-step) and maximization step (M-step) of the algorithm are revised using the conditional probability. Importantly, the closed-form solutions for GMM parameters are rederived in the revised E-step and M-step. Finally, the proposed uncertainty modeling method is compared with the traditional GMM and DPMM on actual wind power and load data from Australia. The proposed method performs the efficiency and accuracy in characterizing uncertainty.

As a powerful nonparametric Bayesian model, the infinite mixture model has been successfully used in machine learning and computer vision. The success of the infinite mixture model owes to the capability clustering and density estimation. In this paper, we propose a nonparametric Bayesian model for single-image super-resolution. Specifically, we combine the Dirichlet process and Gaussian process regression for estimating the distribution of the training patches and modeling the relationship between the low-resolution and high-resolution patches: 1) the proposed method groups the training patches by utilizing the clustering property of Dirichlet process; 2) the proposed method relates the low-resolution and high-resolution patches by predicting the property of Gaussian process; and 3) the mentioned two points are not independent but jointly learned. Hence, the proposed method can make full use of the nonparametric Bayesian model. First, the Dirichlet process mixture model is used to obtain more accurate clusters for training patches. Second, Gaussian process regression is established on each cluster, and this directly reduces the computational complexity. Finally, the two procedures are learned simultaneously to gain the suitable clusters with the ability of prediction. The parameters can be inferred simply via the Gibbs sampling technique. Thorough super-resolution experiments on various images demonstrate that the proposed method is superior to some state-of-the-art methods.

Construction simulation has been widely applied in schedule analysis. However, traditional simulation is based on static models built in the planning or design phase, which focuses on overall project-level schedule analysis. To provide activity-level simulation for on-site schedule management, a construction phase oriented dynamic simulation method is proposed, which takes roller compacted concrete (RCC) dam placement process as an example. Considering various innerlayer and inter-layer activities and different construction organization modes, a detailed placement process simulation model is built. Based on construction data collected by real-time monitoring, a construction activity modeling method is given. Additionally, Dirichlet process mixture (DPM) models are applied for simulation parameter updates, which endows density estimation with considerable flexibility and robustness. A fast inference algorithm is also proposed to realize the fast posterior computation of DPM models. The proposed method is tested by an RCC dam project in southwest China. The results show that the proposed method can reflect the dynamic features of the actual placement process in the construction phase and provide accurate schedule predictions for on-site construction management.

In this work we present and apply a new Hyperspectral image classification method based on Dirichlet Process Gaussian mixture models (also known as infinite Gaussian mixture models - IGMMs). Since this approach is a non-parametric Bayesian method, we circumvent the problem of model selection which is unavoidable and often difficult when employing traditional parametric Gaussian mixture models (GMM). In order to infer model parameters from observations, we adopt a Gibbs Sampling method to sample the posterior distributions of these parameters. In the preprocessing step, we use Local Fisher's Discriminant Analysis (LFDA) for dimension reduction since we expect it to preserve the multi-modal, non-Gaussian structure of the hyperspectral data. We compared our proposed IGMM based classification method to the existing state-of-the-art classification methods using popular hyperspectral imagery datasets. To simulate challenging real-world sensing environment, we mix every pixel with background and study the performance of these classification approaches. Our experiments show that the proposed LFDA-IGMM method and GMM method have almost the same performance for most cases, given “pure” data - with mixed pixels, LFDA-IGMM outperformed GMM for sub-pixel classification. Both methods outperformed the other commonly used classification approaches when the training sample size is relatively large. Also, the IGMM based method is less sensitive to pixel mixing than other methods.

Hidden Markov Random Field (HMRF) models are important statistical tools for image segmentation. However, such models cannot automatically determine the number of clusters. In many practical applications, to determine the number of clusters in advance is impossible. The Dirichlet process mixture (DPM) models are nonparametric Bayesian models which can automatically determine the number of clusters for the clustering problem. A new HMRF model which combines the DPM model with Student's t-distribution is proposed in this paper. Compared to the Gaussian distribution, the Student's t-distribution with heavier-tailed can have more applications. This model can automatically determine the appropriate clustering number from the observed data. In addition, an efficient variational Bayesian inference algorithm is derived to the proposed model. Experimental results show that the proposed model outperforms other DPM-based models in image segmentation.

Multistate models, that is, models with more than two distributions, are preferred over single-state probability models in modeling the distribution of travel time. Literature review indicated that the finite multistate modeling of travel time using lognormal distribution is superior to other probability functions. In this study, we extend the finite multistate lognormal model of estimating the travel time distribution to unbounded lognormal distribution. In particular, a nonparametric Dirichlet Process Mixture Model (DPMM) with stick-breaking process representation was used. The strength of the DPMM is that it can choose the number of components dynamically as part of the algorithm during parameter estimation. To reduce computational complexity, the modeling process was limited to a maximum of six components. Then, the Markov Chain Monte Carlo (MCMC) sampling technique was employed to estimate the parameters’ posterior distribution. Speed data from nine links of a freeway corridor, aggregated on a 5-minute basis, were used to calculate the corridor travel time. The results demonstrated that this model offers significant flexibility in modeling to account for complex mixture distributions of the travel time without specifying the number of components. The DPMM modeling further revealed that freeway travel time is characterized by multistate or single-state models depending on the inclusion of onset and offset of congestion periods.

Finite mixture models have largely been used for providing a convenient format framework for clustering and classification for multivariate data. But most of these models assume that the number of components in mixture model is known in advance. To resolve this issue, we introduce a novel nonparametric Bayesian clustering model, is called Gaussian Dirichlet process mixture model, for the automatic clustering algorithm of multivariate data, and we have also described an efficient variational Bayesian inference algorithm for the proposed model. We apply it to a series of various clustering problems, demonstrating its advantages over existing methodologies.

In this paper, we propose a family of dependent Pitman-Yor (DPY) processes to model the state prior for multiple object tracking. This process is shown to be more flexible and a better match than the dependent Dirichlet process in tracking a time-varying number of objects. The DPY model directly incorporates learning multiple parameters from correlated information. Integrated with a Dirichlet process mixture model, the overall approach estimates time dependent object cardinality, provides object labeling, and identifies object associated measurements. Using Markov chain Monte Carlo sampling methods, the performance of the DPY based approach is demonstrated and compared to the labeled multi Bernoulli tracker.