Federated Truth Inference over Distributed Crowdsourcing Platforms

One effective iterative method for estimating parameters in latent variable probabilistic models is the Expectation-Maximization (EM) algorithm, which finds extensive use in classification issues. A versatile probabilistic framework for modeling data from multiple Gaussian distributions, Gaussian Mixture Models (GMMs) employ EM for parameter estimation, which is explored in this article. Problems with class membership estimation (including means, covariances, and mixing coefficients) are the primary focus of this study. Starting with initial parameter estimations, estimating posterior probabilities in the E-step, and repeatedly updating model parameters in the M-step until convergence, the technique entails applying the EM algorithm on both synthetic and real-world datasets. In order to measure performance, we compare it to K-means and other baseline classifiers using measures like classification accuracy, Adjusted Rand Index (ARI), and log-likelihood progression. When data is distributed according to a multimodal Gaussian distribution, the findings show that EM-GMM provides better parameter estimation and better classification accuracy. Initialization procedures have a substantial effect on performance, as convergence study further shows. In addition to reiterating the EM algorithm's usefulness for probabilistic classification, this paper also describes its advantages and disadvantages and offers suggestions for improving its performance in settings with high-dimensional data and noise.

The traditional Expectation-Maximization (EM) training of Gaussian Mixture Model (GMM) is essentially a batch mode procedure which requires the multiple data samples with the sufficient size to update the model parameters. This severely limits the deployment and adaptation of GMM in many real-time online systems since the newly observed data samples are expected to be incorporated into the system upon available via retraining the model. This paper presents a new online incremental EM training procedure of GMM, which aims to perform the EM training incrementally and so can adapt GMM online sample by sample. The proposed method is extended on two kinds of EM algorithms for GMM, namely, Split-and-Merge EM and the traditional EM. Experiments on both the synthetic data and a speech processing task show the advantages and efficiency of the new method.

As crowdsourcing has cast a new solution to numerous tasks, truth inference, which deduces the accurate answer from massive noise labels (answers), has become quite an essential issue. However, existing proposals of truth inference only excel at limited tasks since they excessively depend on modeling either workers or labels with simple assumptions. In this paper, we propose BAT (Bipartite Attention-driven Truth) to flexibly infer the truth in various scenarios. The key behind BAT is to explore a comprehensive approach from the whole topology of crowdsourcing itself rather than any individual component. Specifically, BAT firstly characterizes the crowdsourcing as an attributed bipartite graph (ABG). Then it deploys a bipartite graph neural network (bi-GNN). The bi-GNN relies on a bipartite attention mechanism for exploiting the importance of different answers to compute the correct one. For verifying BAT, we compare BAT with other eight existing truth inference methods on real-world datasets from different domains (image, text, audio). The results show that BAT performs best on different crowdsourcing tasks.

The expectation maximisation (EM) algorithm is an iterative maximum likelihood procedure often used for estimating the parameters of a mixture model. Theoretically, increases in the likelihood function are guaranteed as the algorithm iteratively improves upon previously derived parameter estimates. The algorithm is considered to converge when all parameter estimates become stable and no further improvements can be made to the likelihood value. However, to reduce computational time, it is often common practice for the algorithm to be stopped before complete convergence using heuristic approaches. In this paper, we consider various stopping criteria and evaluate their effect on fitting Gaussian mixture models (GMMs) to patient length of stay (LOS) data. Although the GMM can be successfully fitted to positively skewed data such as LOS, the fitting procedure often requires many iterations of the EM algorithm. To our knowledge, no previous study has evaluated the effect of different stopping criteria on fitting GMMs to skewed distributions. Hence, the aim of this paper is to evaluate the effect of various stopping criteria in order to select and justify their use within a patient spell classification methodology. Results illustrate that criteria based on the difference in the likelihood value and on the GMM parameters may not always be a good indicator for stopping the algorithm. In fact we show that the values of the difference in the variance parameters should be used instead, as these parameters are the last to stabilise. In addition, we also specify threshold values for the other stopping criteria.

Efficient similarity search for uncertain data is a challenging task in many modern data mining applications like image retrieval, speaker recognition and stock market analysis. A common way to model the uncertainty of data objects is using probability density functions in the form of Gaussian Mixture Models (GMMs), which have an ability to approximate arbitrary distribution. However, due to the possible unequal length of mixture models, the use of existing index techniques has serious problems for the objects modeled by GMMs. Either the techniques cannot handle GMMs or they have too many limitations. Hence, we propose a dynamic index structure, Gaussian Component based Index (GCI), for GMMs. GCI decomposes GMMs into the single, pairs, or n-lets of Gaussian components, stores these components into well studied index trees such as U-tree and Gauss-Tree, and refines the corresponding GMMs in a conservative but tight way. GCI supports both k-most-likely queries and probability threshold queries by means of Matching Probability. Extensive experimental evaluations of GCI demonstrate a considerable speed-up of similarity search on both synthetic and real-world data sets.

The accurate modeling of uncertain variables, such as wind power and load uncertainty, is an essential foundation for the planning operation and scheduling decisions in power systems. Currently, Gaussian mixture model (GMM) and Dirichlet process mixture model (DPMM) are the primary techniques used to characterize uncertainties in power systems, which are commonly solved by expectation-maximization (EM) algorithm. However, for the massive data of uncertain variables, the algorithm encounters challenges in accurately obtaining GMM and DPMM with a lower time consumption. To address this issue, we propose a method for GMM uncertainty modeling in power systems considering the mutual assistance of latent variables. Specifically, the GMM of uncertain variables is first constructed, and the conditional probability is employed to characterize the mutual assistance of latent variables. Then, an improved EM algorithm is developed to obtain the optimal GMM parameters, in which the expectation step (E-step) and maximization step (M-step) of the algorithm are revised using the conditional probability. Importantly, the closed-form solutions for GMM parameters are rederived in the revised E-step and M-step. Finally, the proposed uncertainty modeling method is compared with the traditional GMM and DPMM on actual wind power and load data from Australia. The proposed method performs the efficiency and accuracy in characterizing uncertainty.

In a distribution power network, the load model has no certain pattern or predicted behaviour due to large range of data and changes in energy consumption for end-user consumers. Thus, a powerful analysis based on probabilistic structure is required. For this paper Gaussian Mixture Model (GMM) has been used. GMM is a powerful probability model that allows different types of load distributions to be presented as a combination of several Gaussian distributions. The parameters of GMM is unknown for large random data such as real load data and these parameters can be identified by Expectation Maximization (EM) algorithm. This paper presents a method to evaluate probabilistic load data concerning the time-evolution of any type of distribution load for any duration of time. The proposed method is explained through generated load data of 100 residential houses for duration of one year.

Skeleton extraction from 3D plant point cloud data is an essential prior for myriads of phenotyping studies. Although skeleton extraction from 3D shapes have been studied extensively in the computer vision and graphics literature, handling the case of plants is still an open problem. Drawbacks of the existing approaches include the zigzag structure of the skeleton, nonuniform density of skeleton points, lack of points in the areas having complex geometry structure, and most importantly the lack of biological relevance. With the aim to improve existing skeleton structures of state-of-the-art, we propose a stochastic framework which is supported by the biological structure of the original plant (we consider plants without any leaves). Initially we estimate the branching structure of the plant by the notion of β-splines to form a curve tree defined as a finite set of curves joined in a tree topology with certain level of smoothness. In the next phase, we force the discrete points in the curve tree to move toward the original point cloud by treating each point in the curve tree as a center of Gaussian, and points in the input cloud data as observations from the Gaussians. The task is to find the correct locations of the Gaussian centroids by maximizing a likelihood. The optimization technique is iterative and is based on the Expectation Maximization (EM) algorithm. The E-step estimates which Gaussian the observed point cloud was sampled from, and the M-step maximizes the negative log-likelihood that the observed points were sampled from the Gaussian Mixture Model (GMM) with respect to the model parameters. We experiment with several real world and synthetic datasets and demonstrate the robustness of the approach over the state-of-the-art.

This paper addresses the problem of initialization of the expectation-maximization (EM) algorithm for maximum likelihood estimation of Gaussian mixture models. In order to avoid local maxima of the likelihood function, a genetic algorithm (GA) which searches for best initial conditions of the EM algorithm is proposed. In the GA, a chromosome represents a set of initial conditions, in which initial mean vectors of mixture components are feature vectors chosen from the training set. The chromosome also encodes variances of initial spherical covariance matrices of mixture components. To evaluate each chromosome in the GA we run the EM algorithm until convergence and use the obtained log likelihood as the fitness. In computational experiments our approach was applied to clustering problem and tested on two datasets from the image processing domain. The results indicate that our method outperforms the standard multiple restart EM algorithm and is at least comparable to the state-of-the art random swap EM method.

Segmentation of clusters by template rotation expectation maximization

Gaussian-input Gaussian mixture model for representing density maps and atomic models.

The expectation-maximization (EM) algorithm is a popular approach for parameter estimation of finite mixture model (FMM). A drawback of this approach is that the number of components of the finite mixture model is not known in advance, nevertheless, it is a key issue for EM algorithms. In this paper, a penalized minimum matching distance-guided EM algorithm is discussed. Under the framework of Greedy EM, a fast and accurate algorithm for estimating the number of components of the Gaussian mixture model (GMM) is proposed. The performance of this algorithm is validated via simulative experiments of univariate and bivariate Gaussian mixture models.

Model-based clustering approaches concern the paradigm of exploratory data analysis relying on the finite mixture model to automatically find a latent structure governing observed data. They are one of the most popular and successful approaches in cluster analysis. The mixture density estimation is generally performed by maximizing the observed-data log-likelihood by using the expectation-maximization (EM) algorithm. However, it is well-known that the EM algorithm initialization is crucial. In addition, the standard EM algorithm requires the number of clusters to be known a priori. Some solutions have been provided in [31], [12] for model-based clustering with Gaussian mixture models for multivariate data. In this paper we focus on model-based curve clustering approaches, when the data are curves rather than vectorial data, based on regression mixtures. We propose a new robust EM algorithm for clustering curves. We extend the model-based clustering approach presented in [31] for Gaussian mixture models, to the case of curve clustering by regression mixtures, including polynomial regression mixtures as well as spline or B-spline regressions mixtures. Our approach both handles the problem of initialization and the one of choosing the optimal number of clusters as the EM learning proceeds, rather than in a twofold scheme. This is achieved by optimizing a penalized log-likelihood criterion. A simulation study confirms the potential benefit of the proposed algorithm in terms of robustness regarding initialization and funding the actual number of clusters.

In recent year, stock investor in Indonesia increased rapidly, so it is required to do analysis about the stock that helps the investor in their investment plan. Clustering is beneficial to select the appropriate stock for investors. Unfortunately, stock prices keep varying from time to time. Consequently, it is not an easy work to select the stock for investment. In addition, stock price time series data are high dimensional data that influenced by many factors. In this study, high dimensional data are obtained by the time frame of each factor. Therefore, it is important to use a suitable technique to cluster high dimensional data. This paper presents High Dimensional Data Clustering (HDDC), a model-based clustering based on Gaussian Mixture Model, using the Expectation-Maximization (EM) algorithm. HDDC via EM algorithm gives a more robust result, and it possible to make an additional assumption. Moreover, this paper combines a high-dimensional clustering technique HDDC via EM algorithm and the most popular feature extraction technique Principal Component Analysis (PCA). This paper comparing methods of clustering technique HDDC and the combination between HDDC and PCA to know the most effective method which gives better result in clustering high-dimensional time series data. The 155 data features are reduced to 7 principal components using PCA analysis. Despite PCA has increased the time efficiency of building the model, clustering technique HDDC via EM algorithm enables to handle the high-dimensional data better than the combination with PCA.

Traditional truth inference algorithms take multiple source labels as input and infer true labels for objects. Besides source labels, object features have been introduced in inference algorithms to achieve superior performance. A typical algorithm such as learning from crowds learns a classification model with the guide of inferred true labels where true labels are inferred from source labels. However, the main shortcoming exists in current algorithms and limits their inference performance: label noise. Since source labels from real-world data are noisy, a classifier is likely to be misguided to learn an imprecise decision boundary. In this paper, we propose a deep clustering-based aggregation model (DCAM) to overcome the shortcoming. DCAM introduces clustering for object features to form fine-grained clusters, where objects in the same cluster are supposed to have similar labels. DCAM exploits a cluster label distribution to represent the labeling information of all objects in the corresponding cluster to overcome the problem of label noise. To implement the idea of clustering-based truth inference, DCAM integrates source label generation and deep clustering in a unified framework by utilizing maximum a posteriori (MAP) estimation. Therefore, the proposed model is a novel approach for truth inference with object features. Experimental results on eight real-world inference tasks show that DCAM has a significant improvement of inference accuracy over the state-of-the-art truth inference algorithms. We further discuss the effect of cluster numbers, the quality of clustering, and illustrate the learned embeddings to support the effectiveness of DCAM.