A robust inference algorithm for crowd sourced categorization

Traditional truth inference algorithms take multiple source labels as input and infer true labels for objects. Besides source labels, object features have been introduced in inference algorithms to achieve superior performance. A typical algorithm such as learning from crowds learns a classification model with the guide of inferred true labels where true labels are inferred from source labels. However, the main shortcoming exists in current algorithms and limits their inference performance: label noise. Since source labels from real-world data are noisy, a classifier is likely to be misguided to learn an imprecise decision boundary. In this paper, we propose a deep clustering-based aggregation model (DCAM) to overcome the shortcoming. DCAM introduces clustering for object features to form fine-grained clusters, where objects in the same cluster are supposed to have similar labels. DCAM exploits a cluster label distribution to represent the labeling information of all objects in the corresponding cluster to overcome the problem of label noise. To implement the idea of clustering-based truth inference, DCAM integrates source label generation and deep clustering in a unified framework by utilizing maximum a posteriori (MAP) estimation. Therefore, the proposed model is a novel approach for truth inference with object features. Experimental results on eight real-world inference tasks show that DCAM has a significant improvement of inference accuracy over the state-of-the-art truth inference algorithms. We further discuss the effect of cluster numbers, the quality of clustering, and illustrate the learned embeddings to support the effectiveness of DCAM.

Estimation for the number of components in a mixture model using stepwise split-and-merge EM algorithm

Model-based clustering approaches concern the paradigm of exploratory data analysis relying on the finite mixture model to automatically find a latent structure governing observed data. They are one of the most popular and successful approaches in cluster analysis. The mixture density estimation is generally performed by maximizing the observed-data log-likelihood by using the expectation-maximization (EM) algorithm. However, it is well-known that the EM algorithm initialization is crucial. In addition, the standard EM algorithm requires the number of clusters to be known a priori. Some solutions have been provided in [31], [12] for model-based clustering with Gaussian mixture models for multivariate data. In this paper we focus on model-based curve clustering approaches, when the data are curves rather than vectorial data, based on regression mixtures. We propose a new robust EM algorithm for clustering curves. We extend the model-based clustering approach presented in [31] for Gaussian mixture models, to the case of curve clustering by regression mixtures, including polynomial regression mixtures as well as spline or B-spline regressions mixtures. Our approach both handles the problem of initialization and the one of choosing the optimal number of clusters as the EM learning proceeds, rather than in a twofold scheme. This is achieved by optimizing a penalized log-likelihood criterion. A simulation study confirms the potential benefit of the proposed algorithm in terms of robustness regarding initialization and funding the actual number of clusters.

This work examines the truth inference problem in a distributed crowdsourcing scenario. Labeling tasks are outsourced to workers associated with different platforms, and truth inference is to be performed without sharing the workers’ individual responses with other platforms or the central coordinator. The reliability of the labels may vary over different workers and tasks, and is characterized by a Gaussian mixture model. A federated truth inference (FTI) algorithm is proposed based on a distributed implementation of the block expectation-maximization (EM) algorithm. The messages sent by each local platform to the central coordinator contain aggregates of workers’ responses, instead of individual labels. The convergence of the FTI algorithm can be verified theoretically. A communication-efficient variant of the FTI scheme is also proposed by allowing the distributed platforms to perform multiple local EM computations before updating the global estimates at the central coordinator in each iteration. The effectiveness of our proposed schemes is demonstrated using both synthetic and real-world datasets.

In Recent days Semi supervised image segmentation techniques play a noteworthy role in image processing. Semi supervised image segmentation needs both labeled data and unlabeled data. It means that a Small amount of human assistance or Prior information is given during clustering process. This paper discusses an enhanced semi supervised image segmentation method from labeled image. It uses both a background selection marker and fore ground object selection marker separately. The EM (Expectation Maximization) algorithm is used for clustering along with must link constraints. The proposed method is applied for natural images using MATLAB 7. Thus the proposed method extracts Object of Interest (OOI) from OONI (Object of Not Interest) efficiently and the experimental results are compared with Standard K Means and EM Algorithm also. The results show that the proposed system gives better results than the other two methods. It may also be suitable for object extraction from natural images and medical image analysis.

An image fusion method is proposed based on a kind of estimation theory EM (expectation maximization) algorithm and discrete wavelet frame for merging multiple sensor images. The registered images are firstly decomposed by discrete wavelet frame. Secondly since the fusion of the lowest frequency band is crucial to the final result, they are fused effectively by EM algorithm. The high frequency bands are fused by selection method involving the informative importance measure. Then the fused image is achieved by the discrete wavelet frame invert transform. Experimental results indicate that the proposed method outperforms the discrete wavelet transform and the existing image fusion methods.

Within the context of emission tomography, we study volumetric reconstruction methods based on the Expectation Maximization (EM) algorithm. We show, for the first time, the equivalence of the standard implementation of the EM‐based reconstruction with an implementation based on hardware‐accelerated volume rendering for nearest‐neighbor (NN) interpolation. This equivalence suggests that higher‐order kernels should be used with caution and do not necessarily lead to better performance. We also show that the EM algorithm can easily be adapted for different lattices, the body‐centered cubic (BCC) one in particular. For validation purposes, we use the 3D version of the Shepp‐Logan synthetic phantom, for which we derive closed‐form analytical expressions of the projection data. The experimental results show the theoretically‐predicted optimality of NN interpolation in combination with the EM algorithm, for both the noiseless and the noisy case. Moreover, reconstruction on the BCC lattice leads to superior accuracy, more compact data representation, and better noise reduction compared to the Cartesian one. Finally, we show the usefulness of the proposed method for optical projection tomography of a mouse embryo.

Due to low quality of crowdsourced labelers, the integrated label of each example is usually inferred from its multiple noisy labels provided by different labelers. This paper proposes a novel algorithm, Ground Truth Inference using Clustering (GTIC), to improve the quality of integrated labels for multi-class labeling. For a K labeling case, GTIC utilizes the multiple noisy label sets of examples to generate features. Then, it uses a K-Means algorithm to cluster all examples into K different groups, each of which is mapped to a specific class. Examples in the same cluster are assigned a corresponding class label. We compare GTIC with four existing multi-class ground truth inference algorithms, majority voting (MV), Dawid & Skene's (DS), ZenCrowd (ZC) and Spectral DS (SDS), on one synthetic and eight real-world datasets. Experimental results show that the performance of GTIC is significantly superior to the others in terms of both accuracy and M-AUC. Besides, the running time of GTIC is about twenty times faster than EM-based complicated inference algorithms.

A review of existing multilingual TTS (text-to-speech) systems shows that the secondary language inserted into the primary language sounds more like isolated individual words in an alien language environment and not congruous with the primary language's prosody. Since the letter-by-letter spelling of English words or acronyms appears in Chinese speech quite often, a duration modeling approach for English letters embedded in Chinese speech is proposed to make the English congruous with the primary language's tempo. It takes several major factors as additive factors and estimates all model parameters by an EM (expectation-maximization) algorithm. Experimental results showed that the standard deviation of the duration from the test set was greatly reduced from 59.82 to 9.37 ms by the duration modeling while eliminating effects from factors. The root mean squared error between the original and estimated durations was 9.35 ms for the open tests. Experimental results have confirmed its effectiveness on isolating several main factors that seriously affects the duration. Moreover, the estimated value of the factors agreed well to our prior linguistic knowledge. Besides, the hidden state labels produced by the EM algorithm were linguistically meaningful.

Electronic health records (EHRs) are the records containing the patients' clinic information. The EHRs have been widely used in disease diagnosis and therapy due to the numerous and valuable medical information in them. However, the missing data problem of EHRs hinders the usage. Replacing the missing data with mean values is an approach of data imputation. But, that method weakens the feature importance. In this study, we use the expectation-maximization (EM) algorithm to impute the missing data in EHRs. Some machine learning models, including artificial neural network, logistic regression, support vector machine, and random forests are used to evaluate the effectiveness of data imputation. The experimental results show that the prediction accuracies of cancers by using those models on the EHRs imputed by EM algorithm are higher than those by mean values, which indicates the EM algorithm is able to provide accurate estimations in data imputation of EHRs.

In the scope of image processing expectation maximization (EM) algorithm takes conspicuous place among the other clustering techniques. EM algorithm is suitable for multidimensional data but it requires a number of clusters to be defined a priori that might be a problem for particular applications. The main aim of this paper is to provide time effective EM clustering modification in the case of the unknown number of clusters and multidimensional input. Our work is based on statistical histogram based expectation maximization algorithm (SHEM) proposed by Yang and Huang with the predefined number of clusters. This method utilizes the histogram to provide EM iterations. However, the estimation of the histogram becomes time consuming task with the increase of input data dimension. Our algorithm extends the use of SHEM algorithm by means of a hierarchical histogram data structure, which allows us to reduce the computational load in the multidimensional case as well as to provide an initialization in the case of the unknown number of clusters. The paper includes several experimental results demonstrating the advantages and the disadvantages of the proposed solution

In this paper, we propose a multi-scale Bayesian networks model and its inference algorithm. We use the multi-scale Bayesian networks model to segment the Synthetic Aperture Radar (SAR) image. The multi-scale Bayesian networks is constructed accordance with the multi-scale sequence of SAR images, whose MAP value is performed using the Belief Propagation (BP) algorithm and the corresponding parameter estimation is finished by the Expectation-Maximization (EM) algorithm. Experimental results demonstrate that the proposed multi-scale Bayesian networks model outperform the single-scale Bayesian network model and Markov Random Field - Intersecting Cortical Model (MRF-ICM).

Many important real-world applications of machine learning, statistical physics, constraint programming and information theory can be formulated using graphical models that involve determinism and cycles. Accurate and efficient inference and training of such graphical models remains a key challenge. Markov logic networks (MLNs) have recently emerged as a popular framework for expressing a number of problems which exhibit these properties. While loopy belief propagation (LBP) can be an effective solution in some cases; unfortunately, when both determinism and cycles are present, LBP frequently fails to converge or converges to inaccurate results. As such, sampling based algorithms have been found to be more effective and are more popular for general inference tasks in MLNs. In this paper, we introduce Generalized arc-consistency Expectation Maximization Message-Passing (GEM-MP), a novel message-passing approach to inference in an extended factor graph that combines constraint programming techniques with variational methods. We focus our experiments on Markov logic and Ising models but the method is applicable to graphical models in general. In contrast to LBP, GEM-MP formulates the message-passing structure as steps of variational expectation maximization. Moreover, in the algorithm we leverage the local structures in the factor graph by using generalized arc consistency when performing a variational mean-field approximation. Thus each such update increases a lower bound on the model evidence. Our experiments on Ising grids, entity resolution and link prediction problems demonstrate the accuracy and convergence of GEM-MP over existing state-of-the-art inference algorithms such as MC-SAT, LBP, and Gibbs sampling, as well as convergent message passing algorithms such as the concave---convex procedure, residual BP, and the L2-convex method.

Tractable Bayesian density regression via logit stick-breaking priors

The Probabilistic RBF (PRBF) network constitutes an adaptation of the RBF network for classification. Moreover it extends the typical mixture model by allowing the sharing of mixture components among all classes, in contrast to the conventional approach that suggests mixture components describing only one class. The typical learning method of PRBF for a classification task employs the Expectation – Maximization (EM) algorithm. This widely used method depends strongly on the initial parameter values. The Greedy EM algorithm is a recently proposed method that tries to overcome this drawback, in the case of the density estimation problem using mixture models. In this work we propose a similar approach for incremental training of the PRBF network for classification. The proposed algorithm starts with a single component and incrementally adds more components. After convergence the algorithm splits all the components of the network. The addition of a new component is based on criteria for detecting a region in the data space that is crucial for the classification task. Experimental results using several well-known classification datasets indicate that the incremental method provides solutions of superior classification performance.