A study on probability of distribution loads based on expectation maximization algorithm

The accurate modeling of uncertain variables, such as wind power and load uncertainty, is an essential foundation for the planning operation and scheduling decisions in power systems. Currently, Gaussian mixture model (GMM) and Dirichlet process mixture model (DPMM) are the primary techniques used to characterize uncertainties in power systems, which are commonly solved by expectation-maximization (EM) algorithm. However, for the massive data of uncertain variables, the algorithm encounters challenges in accurately obtaining GMM and DPMM with a lower time consumption. To address this issue, we propose a method for GMM uncertainty modeling in power systems considering the mutual assistance of latent variables. Specifically, the GMM of uncertain variables is first constructed, and the conditional probability is employed to characterize the mutual assistance of latent variables. Then, an improved EM algorithm is developed to obtain the optimal GMM parameters, in which the expectation step (E-step) and maximization step (M-step) of the algorithm are revised using the conditional probability. Importantly, the closed-form solutions for GMM parameters are rederived in the revised E-step and M-step. Finally, the proposed uncertainty modeling method is compared with the traditional GMM and DPMM on actual wind power and load data from Australia. The proposed method performs the efficiency and accuracy in characterizing uncertainty.

The expectation maximisation (EM) algorithm is an iterative maximum likelihood procedure often used for estimating the parameters of a mixture model. Theoretically, increases in the likelihood function are guaranteed as the algorithm iteratively improves upon previously derived parameter estimates. The algorithm is considered to converge when all parameter estimates become stable and no further improvements can be made to the likelihood value. However, to reduce computational time, it is often common practice for the algorithm to be stopped before complete convergence using heuristic approaches. In this paper, we consider various stopping criteria and evaluate their effect on fitting Gaussian mixture models (GMMs) to patient length of stay (LOS) data. Although the GMM can be successfully fitted to positively skewed data such as LOS, the fitting procedure often requires many iterations of the EM algorithm. To our knowledge, no previous study has evaluated the effect of different stopping criteria on fitting GMMs to skewed distributions. Hence, the aim of this paper is to evaluate the effect of various stopping criteria in order to select and justify their use within a patient spell classification methodology. Results illustrate that criteria based on the difference in the likelihood value and on the GMM parameters may not always be a good indicator for stopping the algorithm. In fact we show that the values of the difference in the variance parameters should be used instead, as these parameters are the last to stabilise. In addition, we also specify threshold values for the other stopping criteria.

Salient object detection in a real-time environment demands high accuracy with less computation time. It is a changeling task to investigate a saliency model which improves saliency detection accuracy as well as reduces computation time. In this paper, we propose a hybrid model that improves detection accuracy with low computational time. The input image is simplified and clustered at multiple scales using SLIC and K-means clustering algorithms respectively. Gaussian Mixture Model (GMM) is developed on various color components of the digital image at multiple scales. The parameters of GMM are learnt using Expectation Maximization (EM) algorithm. The spatial variance of each color component is determined using GMM parameters and hence object position is estimated. Further, spatial variance of color components and object position is exploited to compute saliency map at a scale level. Afterwards, all the saliency maps generated across various scales are linearly combined to produce the final saliency map. The performance of the proposed model is compared in terms of Precision, Recall, F-Measure, Area under the Curve (AUC), Receiver Operating Characteristics (ROC) and Mean Absolute Error (MAE). Extensive experiments on six publicly available datasets viz. MSRA10K, DUT-OMRON, ECSSD, PASCAL-S, SED2, and THUR15K show that the proposed model outperforms or comparable against 11 state-of-the-art methods of the last decade. The key features of the proposed method are object completeness and efficiency in terms of computational time.

In this paper we apply generative modeling to Gaussian Mixture Models (GMM) as a solution to high dimensional novel event detection and analysis on radio frequency (RF) power generators. A family of Denoising and β-Disentangling Variational Autoencoders β-DVAE) is used to encode datasets into lower and more salient dimensions. The reduced feature sets are modeled using GMMs where model parameters are learned using the Expectation Maximization (EM) algorithm. The data is obtained from two different generator models operating under normal as well as known not-normal conditions. This approach is also tested on standard classification sets. Robust testing is reported to achieve a target class accuracy of 98.16% for the target RF generator. Additionally, the GMM parameters are decoded by the generative model to the original data space for calculating per-variable fitness in order to provide an initial novel event analysis for engineers and technicians. This per-variable fitness is very critical for determining each variable's contribution to a novelty so that engineers can perform informed trouble shooting and maintenance of the RF generators.

One effective iterative method for estimating parameters in latent variable probabilistic models is the Expectation-Maximization (EM) algorithm, which finds extensive use in classification issues. A versatile probabilistic framework for modeling data from multiple Gaussian distributions, Gaussian Mixture Models (GMMs) employ EM for parameter estimation, which is explored in this article. Problems with class membership estimation (including means, covariances, and mixing coefficients) are the primary focus of this study. Starting with initial parameter estimations, estimating posterior probabilities in the E-step, and repeatedly updating model parameters in the M-step until convergence, the technique entails applying the EM algorithm on both synthetic and real-world datasets. In order to measure performance, we compare it to K-means and other baseline classifiers using measures like classification accuracy, Adjusted Rand Index (ARI), and log-likelihood progression. When data is distributed according to a multimodal Gaussian distribution, the findings show that EM-GMM provides better parameter estimation and better classification accuracy. Initialization procedures have a substantial effect on performance, as convergence study further shows. In addition to reiterating the EM algorithm's usefulness for probabilistic classification, this paper also describes its advantages and disadvantages and offers suggestions for improving its performance in settings with high-dimensional data and noise.

The traditional Expectation-Maximization (EM) training of Gaussian Mixture Model (GMM) is essentially a batch mode procedure which requires the multiple data samples with the sufficient size to update the model parameters. This severely limits the deployment and adaptation of GMM in many real-time online systems since the newly observed data samples are expected to be incorporated into the system upon available via retraining the model. This paper presents a new online incremental EM training procedure of GMM, which aims to perform the EM training incrementally and so can adapt GMM online sample by sample. The proposed method is extended on two kinds of EM algorithms for GMM, namely, Split-and-Merge EM and the traditional EM. Experiments on both the synthetic data and a speech processing task show the advantages and efficiency of the new method.

In this paper, the Gaussian mixture model (GMM) is introduced to implement channel multipath clustering. The GMM incorporates the covariance structure and the mean information of the channel multipaths, thus it can effectively reveal the similarity of the channel multipaths. First, the expectation-maximization (EM) algorithm is utilized to search for the posterior estimation of the GMM parameters. Then, the variational Bayesian (VB) algorithm is employed to optimize the GMM parameters to enhance the searching ability of EM and further to determine the optimal number of Gaussian distributions without resorting to cross-validation. Finally, a compact index (CI) is proposed to validate the clustering results reasonably. Thanks to the proposed CI, it is possible to find a close relationship among the GMM clustering mechanism, the multipath propagation characteristics and the CI evaluation index. Experiments with synthetic data and outdoor-to-indoor (O2I) channel data are presented to demonstrate the effectiveness of the proposed method.

The Expectation-Maximization algorithm is a very popular approach for estimating the parameters of Gaussian mixture models (GMMs). A known issue with GMM estimation is its sensitivity to outliers, which can lead to poor estimation performance depending on the dataset under consideration. A common approach to deal with this issue is robust estimation, which typically consists of reducing the influence of the outliers on the estimators by weighting the impact of some samples of the dataset considered as outliers. In an unsupervised context, it is difficult to know which sample from the database corresponds to a normal observation. To that extent, we propose to use within the EM algorithm an outlier detection step that attributes an anomaly score to each sample of the database in an unsupervised way. A modified Bayesian Information Criterion is also introduced to efficiently select the appropriate amount of outliers contained in a dataset. The proposed method is tested on a benchmark remote sensing dataset coming from the UCI Machine Learning Repository. The experimental results show the interest of the proposed robustification when compared to other benchmark imputation procedures.

Intelligent vehicles need reliable information about the environment in order to operate with total safety. In this paper we propose a flexible multi-module architecture for a multi-target detection and tracking system (MTDTS) complemented with a Bayesian object classification layer based on finite Gaussian mixture models (GMM). The GMM parameters are estimated by an expectation maximization (EM) algorithm, hence finite-component models were generated based on feature-vectors extracted from object's classes during the training stage. Using the joint mixture Gaussian pdf modelled for each class, a Bayesian approach is used to distinct the object's categories (persons, tree-trunks/posts, and cars) in a semi-structured outdoor environment based on data from a laser range finder (LRF). Experiments using real data scan confirm the robustness of the proposed architecture. This paper investigates a particular problem: detection, tracking and classification of objects in cybercars-like outdoor environments

Condition assessment of cables by pattern recognition of vehicle-induced cable tension ratio

We present a novel approach for prediction of satellite image frame that uses a fuzzy rule based framework. The input-output membership functions for the premise and consequent parts of the rules are derived using a Gaussian Mixture Model (GMM). The weights of the fuzzy rules are represented as the prior probabilities of the respective Gaussian components. For obtaining the predictive fuzzy model, the GMM parameters are estimated via EM algorithm using a spatiotemporal representation of image sequence or video clips. Minimum Description Length (MDL) criterion is used to obtain a suitable predictive fuzzy model. The resulting model is successfully applied on a sequence of satellite images of tropical cyclone, Nargis, that made landfall in Myanmar on May 2, 2008. The quality of the predicted image is assessed using two criteria. The proposed approach is found to predict image frame successfully.

The accurate reconstruction of vehicle paths is essential for effective highway toll management. To address the challenge of multiple possible paths due to missing trajectory data, this study proposes a novel two-stage model for vehicle path reconstruction. In the first stage, a Gaussian Mixture Model (GMM) is integrated into a path choice model to estimate the mean and standard deviation of travel times for each road segment, utilizing an improved Expectation Maximization (EM) algorithm. In the second stage, based on the estimated time parameters, path choice prior probabilities and observed data are combined using maximum likelihood estimation to infer the most probable paths among candidate routes. The results indicate that the improved EM algorithm achieved convergence in 17 iterations compared to 41 iterations for the traditional EM algorithm. The two-stage model outperforms the Shortest Path and Bidirectional Long Short-Term Memory models in path reconstruction, particularly with a high number of missing trajectory points. Additionally, when the number of candidate paths K=4, the path reconstruction performance is optimal. These results demonstrate the effectiveness of the proposed method in handling sparse and incomplete trajectory data, offering robust and accurate vehicle path estimations that enhance traffic management and toll calculation precision.

EM algorithm is a very popular iteration-based method to estimate the parameters of Gaussian Mixture Model from a large observation set. However, in most cases, EM algorithm is not guaranteed to converge to the global optimum. Instead, it stops at some local optimums, which can be much worse than the global optimum. Therefore, it is usually required to run multiple procedures of EM algorithm with different initial configurations and return the best solution. To improve the efficiency of this scheme, we propose a new method which can estimate an upper bound on the logarithm likelihood of the local optimum, based on the current configuration after the latest EM iteration. This is accomplished by first deriving some region bounding the possible locations of local optimum, followed by some upper bound estimation on the maximum likelihood. With this estimation, we can terminate an EM algorithm procedure if the estimated local optimum is definitely worse than the best solution seen so far. Extensive experiments show that our method can effectively and efficiently accelerate conventional multiple restart EM algorithm.

This paper addresses the problem of initialization of the expectation-maximization (EM) algorithm for maximum likelihood estimation of Gaussian mixture models. In order to avoid local maxima of the likelihood function, a genetic algorithm (GA) which searches for best initial conditions of the EM algorithm is proposed. In the GA, a chromosome represents a set of initial conditions, in which initial mean vectors of mixture components are feature vectors chosen from the training set. The chromosome also encodes variances of initial spherical covariance matrices of mixture components. To evaluate each chromosome in the GA we run the EM algorithm until convergence and use the obtained log likelihood as the fitness. In computational experiments our approach was applied to clustering problem and tested on two datasets from the image processing domain. The results indicate that our method outperforms the standard multiple restart EM algorithm and is at least comparable to the state-of-the art random swap EM method.

Segmentation of clusters by template rotation expectation maximization