A Local Scalable Distributed Expectation Maximization Algorithm for Large Peer-to-Peer Networks

There has been growing interest in performing signal processing tasks directly on compressive measurements, e.g. low-dimensional linear measurements of signals taken with Gaussian random vectors. In this paper, we present a highly efficient algorithm to recover the covariance matrix of high-dimensional data from compressive measurements. We show that, as the number of data samples increases, the eigenvectors (principal components) of the empirical covariance matrix of a simple matrix-vector multiplication of the compressive measurements converge to the true principal components of the original data. Also, we investigate the perturbation of eigenvalues of the covariance matrix under random projection of the data to find conditions for approximate recovery of them. Furthermore, we introduce an important application of our proposed method for efficient estimation of the parameters of Gaussian Mixture Models from compressive measurements. We present experimental results demonstrating the performance and efficiency of our proposed algorithms.

The Expectation-Maximization algorithm is a very popular approach for estimating the parameters of Gaussian mixture models (GMMs). A known issue with GMM estimation is its sensitivity to outliers, which can lead to poor estimation performance depending on the dataset under consideration. A common approach to deal with this issue is robust estimation, which typically consists of reducing the influence of the outliers on the estimators by weighting the impact of some samples of the dataset considered as outliers. In an unsupervised context, it is difficult to know which sample from the database corresponds to a normal observation. To that extent, we propose to use within the EM algorithm an outlier detection step that attributes an anomaly score to each sample of the database in an unsupervised way. A modified Bayesian Information Criterion is also introduced to efficiently select the appropriate amount of outliers contained in a dataset. The proposed method is tested on a benchmark remote sensing dataset coming from the UCI Machine Learning Repository. The experimental results show the interest of the proposed robustification when compared to other benchmark imputation procedures.

In this paper we seek a Gaussian mixture model (GMM) of an n-variate probability density function. Usually the parameters of GMMs are determined in the original n-dimensional space by optimizing a maximum likelihood (ML) criterion. A practical deficiency of this method of fitting GMMs is its poor performance when dealing with high-dimensional data since a large sample size is needed to match the accuracy that is possible in low dimensions. We propose a method for fitting the GMM based on the projection pursuit strategy. This GMM is highly constrained and hence its ability to model structure in subspaces is enhanced, compared to a direct ML fitting of a GMM in high dimensions. Our method is closely related to recently developed independent factor analysis (IFA) mixture models. The comparisons with ML fitting of GMM in n-dimensions and IFA mixtures show that the proposed method is an attractive choice for fitting GMMs using small sizes of training sets.

The expectation maximisation (EM) algorithm is an iterative maximum likelihood procedure often used for estimating the parameters of a mixture model. Theoretically, increases in the likelihood function are guaranteed as the algorithm iteratively improves upon previously derived parameter estimates. The algorithm is considered to converge when all parameter estimates become stable and no further improvements can be made to the likelihood value. However, to reduce computational time, it is often common practice for the algorithm to be stopped before complete convergence using heuristic approaches. In this paper, we consider various stopping criteria and evaluate their effect on fitting Gaussian mixture models (GMMs) to patient length of stay (LOS) data. Although the GMM can be successfully fitted to positively skewed data such as LOS, the fitting procedure often requires many iterations of the EM algorithm. To our knowledge, no previous study has evaluated the effect of different stopping criteria on fitting GMMs to skewed distributions. Hence, the aim of this paper is to evaluate the effect of various stopping criteria in order to select and justify their use within a patient spell classification methodology. Results illustrate that criteria based on the difference in the likelihood value and on the GMM parameters may not always be a good indicator for stopping the algorithm. In fact we show that the values of the difference in the variance parameters should be used instead, as these parameters are the last to stabilise. In addition, we also specify threshold values for the other stopping criteria.

In this paper we apply generative modeling to Gaussian Mixture Models (GMM) as a solution to high dimensional novel event detection and analysis on radio frequency (RF) power generators. A family of Denoising and β-Disentangling Variational Autoencoders β-DVAE) is used to encode datasets into lower and more salient dimensions. The reduced feature sets are modeled using GMMs where model parameters are learned using the Expectation Maximization (EM) algorithm. The data is obtained from two different generator models operating under normal as well as known not-normal conditions. This approach is also tested on standard classification sets. Robust testing is reported to achieve a target class accuracy of 98.16% for the target RF generator. Additionally, the GMM parameters are decoded by the generative model to the original data space for calculating per-variable fitness in order to provide an initial novel event analysis for engineers and technicians. This per-variable fitness is very critical for determining each variable's contribution to a novelty so that engineers can perform informed trouble shooting and maintenance of the RF generators.

The accurate modeling of uncertain variables, such as wind power and load uncertainty, is an essential foundation for the planning operation and scheduling decisions in power systems. Currently, Gaussian mixture model (GMM) and Dirichlet process mixture model (DPMM) are the primary techniques used to characterize uncertainties in power systems, which are commonly solved by expectation-maximization (EM) algorithm. However, for the massive data of uncertain variables, the algorithm encounters challenges in accurately obtaining GMM and DPMM with a lower time consumption. To address this issue, we propose a method for GMM uncertainty modeling in power systems considering the mutual assistance of latent variables. Specifically, the GMM of uncertain variables is first constructed, and the conditional probability is employed to characterize the mutual assistance of latent variables. Then, an improved EM algorithm is developed to obtain the optimal GMM parameters, in which the expectation step (E-step) and maximization step (M-step) of the algorithm are revised using the conditional probability. Importantly, the closed-form solutions for GMM parameters are rederived in the revised E-step and M-step. Finally, the proposed uncertainty modeling method is compared with the traditional GMM and DPMM on actual wind power and load data from Australia. The proposed method performs the efficiency and accuracy in characterizing uncertainty.

In this paper we investigate the problem of automatically identifying the genre of TV programmes. The approach here proposed is based on two foundations: Gaussian mixture models (GMMs) and artificial neural networks (ANNs). Firstly, we use Gaussian mixtures to model the probability distributions of low-level audiovisual features. Secondly, we use the parameters of each mixture model as new feature vectors. Finally, we train a multilayer perceptron (MLP), using GMM parameters as input data, to identify seven television programme genres. We evaluated the effectiveness of the proposed approach testing our system on a large set of data, summing up to more than 100 hours of broadcasted programmes.

EM algorithm is a very popular iteration-based method to estimate the parameters of Gaussian Mixture Model from a large observation set. However, in most cases, EM algorithm is not guaranteed to converge to the global optimum. Instead, it stops at some local optimums, which can be much worse than the global optimum. Therefore, it is usually required to run multiple procedures of EM algorithm with different initial configurations and return the best solution. To improve the efficiency of this scheme, we propose a new method which can estimate an upper bound on the logarithm likelihood of the local optimum, based on the current configuration after the latest EM iteration. This is accomplished by first deriving some region bounding the possible locations of local optimum, followed by some upper bound estimation on the maximum likelihood. With this estimation, we can terminate an EM algorithm procedure if the estimated local optimum is definitely worse than the best solution seen so far. Extensive experiments show that our method can effectively and efficiently accelerate conventional multiple restart EM algorithm.

The study deals with the challenging task of automatic segmentation of MR brain images with multiple sclerosis lesions (MSL). Multi-Channel data is used, including "fast fluid attenuated inversion recovery" (fast FLAIR or FF), and statistical modeling tools are developed, in order to improve cerebrospinal fluid (CSF) classification and to detect MSL. Two new concepts are proposed for use within an EM framework. The first concept is the integration of prior knowledge as it relates to tissue behavior in different MRI modalities, with special attention given to the FF modality. The second concept deals with running the algorithm on a subset of the input that is most likely to be noise- and artifact-free data. This enables a more reliable learning of the Gaussian mixture model (GMM) parameters for brain tissue statistics. The proposed method focuses on the problematic CSF intensity distribution, which is a key to improved overall segmentation and lesion detection. A level-set based active contour stage is performed for lesion delineation, using gradient and shape properties combined with previously learned region intensity statistics. In the proposed scheme there is no need for preregistration of an atlas, a common characteristic in brain segmentation schemes. Experimental results on real data are presented.

We study cooperative sensor network localization in a realistic scenario where the underlying measurement errors more probably follow a non-Gaussian distribution; the measurement error distribution is unknown without conducting massive offline calibrations; and non-line-of-sight identification is not performed due to the complexity constraint and/or storage limitation. The underlying measurement error distribution is approximated parametrically by a Gaussian mixture with finite number of components, and the expectation-conditional maximization (ECM) criterion is adopted to approximate the maximum-likelihood estimator of the unknown sensor positions and an extra set of Gaussian mixture model parameters. The resulting centralized ECM algorithms lead to easier inference tasks and meanwhile retain several convergence properties with a proof of the “space filling” condition. To meet the scalability requirement, we further develop two distributed ECM algorithms where an average consensus algorithm plays an important role for updating the Gaussian mixture model parameters locally. The proposed algorithms are analyzed systematically in terms of computational complexity and communication overhead. Various computer based tests are also conducted with both simulation and experimental data. The results pin down that the proposed distributed algorithms can provide overall good performance for the assumed scenario even under model mismatch, while the existing competing algorithms either cannot work without the prior knowledge of the measurement error statistics or merely provide degraded localization performance when the measurement error is clearly non-Gaussian.

In this paper, an image stabilization method based on the adaptive Gaussian mixture model (GMM) is presented in order to smooth down the airborne image vibration. Firstly, the projection algorithm is adopted for the motion estimation; secondly, GMM parameter is obtained after analyzing characteristics of the first n images; finally, a stable image sequence is achieved after GMM motion filter operates on the motion parameter. The experimental results show that the method has the advantage of fast speed and effectively smooth unwanted vibration of image sequences.

In this paper, the Gaussian mixture model (GMM) is introduced to implement channel multipath clustering. The GMM incorporates the covariance structure and the mean information of the channel multipaths, thus it can effectively reveal the similarity of the channel multipaths. First, the expectation-maximization (EM) algorithm is utilized to search for the posterior estimation of the GMM parameters. Then, the variational Bayesian (VB) algorithm is employed to optimize the GMM parameters to enhance the searching ability of EM and further to determine the optimal number of Gaussian distributions without resorting to cross-validation. Finally, a compact index (CI) is proposed to validate the clustering results reasonably. Thanks to the proposed CI, it is possible to find a close relationship among the GMM clustering mechanism, the multipath propagation characteristics and the CI evaluation index. Experiments with synthetic data and outdoor-to-indoor (O2I) channel data are presented to demonstrate the effectiveness of the proposed method.

In a distribution power network, the load model has no certain pattern or predicted behaviour due to large range of data and changes in energy consumption for end-user consumers. Thus, a powerful analysis based on probabilistic structure is required. For this paper Gaussian Mixture Model (GMM) has been used. GMM is a powerful probability model that allows different types of load distributions to be presented as a combination of several Gaussian distributions. The parameters of GMM is unknown for large random data such as real load data and these parameters can be identified by Expectation Maximization (EM) algorithm. This paper presents a method to evaluate probabilistic load data concerning the time-evolution of any type of distribution load for any duration of time. The proposed method is explained through generated load data of 100 residential houses for duration of one year.

Foreground detection can be considered as backbone of multistage computer vision systems. Foreground detection using Gaussian Mixture Models (GMM) is famous choice because of its good accuracy and low computational cost. There are several parameters (e.g., learning rate, mean, and variance) involved in the model and assigning appropriate values may lead to better foreground segmentation. This paper analyzes the effect of different parameters of GMM on the extraction of foreground. Furthermore, optimal parameter setting suitable in every background setting e.g. indoor, outdoor, and complex backgrounds, etc. is proposed. Standard datasets with indoor and outdoor sequences were tested.

Salient object detection in a real-time environment demands high accuracy with less computation time. It is a changeling task to investigate a saliency model which improves saliency detection accuracy as well as reduces computation time. In this paper, we propose a hybrid model that improves detection accuracy with low computational time. The input image is simplified and clustered at multiple scales using SLIC and K-means clustering algorithms respectively. Gaussian Mixture Model (GMM) is developed on various color components of the digital image at multiple scales. The parameters of GMM are learnt using Expectation Maximization (EM) algorithm. The spatial variance of each color component is determined using GMM parameters and hence object position is estimated. Further, spatial variance of color components and object position is exploited to compute saliency map at a scale level. Afterwards, all the saliency maps generated across various scales are linearly combined to produce the final saliency map. The performance of the proposed model is compared in terms of Precision, Recall, F-Measure, Area under the Curve (AUC), Receiver Operating Characteristics (ROC) and Mean Absolute Error (MAE). Extensive experiments on six publicly available datasets viz. MSRA10K, DUT-OMRON, ECSSD, PASCAL-S, SED2, and THUR15K show that the proposed model outperforms or comparable against 11 state-of-the-art methods of the last decade. The key features of the proposed method are object completeness and efficiency in terms of computational time.