Modeling and evaluating Gaussian mixture model based on motion granularity

The traditional Expectation-Maximization (EM) training of Gaussian Mixture Model (GMM) is essentially a batch mode procedure which requires the multiple data samples with the sufficient size to update the model parameters. This severely limits the deployment and adaptation of GMM in many real-time online systems since the newly observed data samples are expected to be incorporated into the system upon available via retraining the model. This paper presents a new online incremental EM training procedure of GMM, which aims to perform the EM training incrementally and so can adapt GMM online sample by sample. The proposed method is extended on two kinds of EM algorithms for GMM, namely, Split-and-Merge EM and the traditional EM. Experiments on both the synthetic data and a speech processing task show the advantages and efficiency of the new method.

In this paper we applied the optimal mixture number estimation technique in GMM(Gaussian mixture model) using BIC(Bayesian information criterion) and MDL(minimum description length) as a model selection criterion for environmental sounds recognition. In the experiment, we extracted 12 MFCC(mel-frequency cepstral coefficients) features from 9 kinds of environmental sounds which amounts to 27747 data and classified them with GMM. As mentioned above, BIC and MDL is applied to estimate the optimal number of mixtures in each environmental sounds class. According to the experimental results, while the recognition performances are maintained, the computational complexity decreases by 17.8% with BIC and 31.7% with MDL. It shows that the computational complexity reduction by BIC and MDL is effective for environmental sounds recognition using GMM.

In recent year, stock investor in Indonesia increased rapidly, so it is required to do analysis about the stock that helps the investor in their investment plan. Clustering is beneficial to select the appropriate stock for investors. Unfortunately, stock prices keep varying from time to time. Consequently, it is not an easy work to select the stock for investment. In addition, stock price time series data are high dimensional data that influenced by many factors. In this study, high dimensional data are obtained by the time frame of each factor. Therefore, it is important to use a suitable technique to cluster high dimensional data. This paper presents High Dimensional Data Clustering (HDDC), a model-based clustering based on Gaussian Mixture Model, using the Expectation-Maximization (EM) algorithm. HDDC via EM algorithm gives a more robust result, and it possible to make an additional assumption. Moreover, this paper combines a high-dimensional clustering technique HDDC via EM algorithm and the most popular feature extraction technique Principal Component Analysis (PCA). This paper comparing methods of clustering technique HDDC and the combination between HDDC and PCA to know the most effective method which gives better result in clustering high-dimensional time series data. The 155 data features are reduced to 7 principal components using PCA analysis. Despite PCA has increased the time efficiency of building the model, clustering technique HDDC via EM algorithm enables to handle the high-dimensional data better than the combination with PCA.

Gaussian-input Gaussian mixture model for representing density maps and atomic models.

In a distribution power network, the load model has no certain pattern or predicted behaviour due to large range of data and changes in energy consumption for end-user consumers. Thus, a powerful analysis based on probabilistic structure is required. For this paper Gaussian Mixture Model (GMM) has been used. GMM is a powerful probability model that allows different types of load distributions to be presented as a combination of several Gaussian distributions. The parameters of GMM is unknown for large random data such as real load data and these parameters can be identified by Expectation Maximization (EM) algorithm. This paper presents a method to evaluate probabilistic load data concerning the time-evolution of any type of distribution load for any duration of time. The proposed method is explained through generated load data of 100 residential houses for duration of one year.

The accurate modeling of uncertain variables, such as wind power and load uncertainty, is an essential foundation for the planning operation and scheduling decisions in power systems. Currently, Gaussian mixture model (GMM) and Dirichlet process mixture model (DPMM) are the primary techniques used to characterize uncertainties in power systems, which are commonly solved by expectation-maximization (EM) algorithm. However, for the massive data of uncertain variables, the algorithm encounters challenges in accurately obtaining GMM and DPMM with a lower time consumption. To address this issue, we propose a method for GMM uncertainty modeling in power systems considering the mutual assistance of latent variables. Specifically, the GMM of uncertain variables is first constructed, and the conditional probability is employed to characterize the mutual assistance of latent variables. Then, an improved EM algorithm is developed to obtain the optimal GMM parameters, in which the expectation step (E-step) and maximization step (M-step) of the algorithm are revised using the conditional probability. Importantly, the closed-form solutions for GMM parameters are rederived in the revised E-step and M-step. Finally, the proposed uncertainty modeling method is compared with the traditional GMM and DPMM on actual wind power and load data from Australia. The proposed method performs the efficiency and accuracy in characterizing uncertainty.

This study presents an application of the Gaussian mixture (GM) method for soil classification using multiple cone penetration tests (CPT). Compared to the hard clustering methods, the GM model classifies the CPT data by representing the probability density function of observed variables as a mixture of multivariate normal distributions. A GM model based expectation maximization (EM) algorithm with Bayesian information criterion (BIC) for selecting the optimal number of clusters is developed using six real CPT data performed at Dunkerque site in the north of France. The classification results are compared with the classical CPT based interpretation using the non normalized soil behavior type (SBT) index together with the Robertson chart. The results show that the GM model is able to identify accurately the soil layers. In addition, the combination of all CPTs, rather than considering them separately, may improve the soil layers identification because all the site information is considered.

A blind event-based learning algorithm for non-intrusive load disaggregation

In the era of data-driven decision-making, the Gaussian Mixture Model (GMM) stands as a cornerstone in statistical modeling, particularly in clustering and density estimation. The Improved GMM presents a robust solution to a fundamental problem in clustering: the determination of the optimal number of clusters. Unlike its predecessor, it does not rely on a predetermined cluster count but employs model selection criteria, such as the Bayesian Information Criterion (BIC) or Akaike Information Criterion (AIC), to automatically identify the most suitable cluster count for the given data. This inherent adaptability is a hallmark of the Improved GMM, making it a versatile tool in a broad spectrum of applications, from market segmentation to image processing. Furthermore, the Improved GMM revolutionizes parameter estimation and model fitting. It leverages advanced optimization techniques, such as the Expectation-Maximization (EM) algorithm or variational inference, to achieve convergence to more favorable local optima. This results in precise and reliable parameter estimates, including cluster means, covariances, and component weights. The Improved GMM is particularly invaluable when dealing with data of varying complexities, non-standard data distributions, and clusters with differing shapes and orientations. It excels at capturing the nuanced relationships within the data, providing a powerful framework for understanding complex systems. One of the key differentiators of the Improved GMM is its accommodation of full covariance matrices for each component. This feature empowers the model to account for intricate interdependencies between variables, which is essential for modeling real-world data effectively. It is capable of handling data that exhibits non-spherical or irregular cluster shapes, a significant limitation of the traditional GMM.

This article proposes the distribution similarity measure–based Gaussian mixtures model for the contact-state (CS) modelling in force-guided robotic assembly processes of flexible rubber parts. The wrench (Cartesian force and torque) signals of the manipulated object are captured for different states of the given assembly process. The distribution similarity measure–based Gaussian mixtures model CS modelling scheme is employed in modelling the captured wrench signals for different CSs. The proposed distribution similarity measure–based Gaussian mixtures model CS modelling scheme uses the Gaussian mixtures model in modelling the captured signals. The parameters of the Gaussian mixtures models are computed using expectation maximisation. The optimal number of Gaussian mixtures model components for each CS model is determined by considering the classification success rate as an index for the similarity measure between the distribution of the captured signals and the developed models. The optimal number of Gaussian mixtures model components corresponds to the highest classification success rate; hence, object elasticity variation would be accommodated by properly choosing the optimal number of Gaussian mixtures model components. The performance of the proposed distribution similarity measure–based Gaussian mixtures model CS modelling strategy is evaluated by a test stand composed of a KUKA lightweight robot doing peg-in-hole assembly processes for flexible rubber objects. Two rubber objects with different elasticity are considered for two experiments; in the first experiment, an elastic peg of 30 Shore A hardness is considered and that of the second experiment has hardness of 6 Shore A which is even softer than the one used in experiment 1. Employing the proposed distribution similarity measure–based Gaussian mixtures model CS modelling strategy excellent classification success rate was obtained for both experiments. However, more Gaussian mixtures model components are required for the softer one that gives a strong impression of the non-stationarity behaviour increment for softer materials. Comparison is performed with the available CS modelling schemes and the distribution similarity measure–based Gaussian mixtures model is shown to provide the best classification success rate performance with a reduced computational time.

Using Gaussian mixture models for clustering is a statistically mature method for clustering in data science with numerous successful applications in science and engineering. The parameters for a Gaussian mixture model are typically estimated from training data using the iterative expectation-maximization algorithm, which requires the number of Gaussian components a priori. In this study, we propose two algorithms rooted in numerical algebraic geometry, namely an area-based algorithm and a local maxima algorithm, to identify the optimal number of components. The area-based algorithm transforms several Gaussian mixture models with varying number of components into sets of equivalent polynomial regression splines. Next, it uses homotopy continuation methods for evaluating the resulting splines to identify the number of components that results in the best fit. The local maxima algorithm forms a set of polynomials by fitting a smoothing spline to a kernel density estimate of the data. Next, it uses numerical algebraic geometry to solve the system of the first derivatives for finding the local maxima of the resulting smoothing spline, which estimates the number of mixture components. The local maxima algorithm also identifies the location of the centers of Gaussian components. Using a real-world case study in automotive manufacturing and multiple simulations, we compare the performance of the proposed algorithms with that of Akaike information criterion (AIC) and Bayesian information criterion (BIC), which are popular methods in the literature. We show the proposed algorithms are more robust than AIC and BIC when the Gaussian assumption is violated.

The expectation maximisation (EM) algorithm is an iterative maximum likelihood procedure often used for estimating the parameters of a mixture model. Theoretically, increases in the likelihood function are guaranteed as the algorithm iteratively improves upon previously derived parameter estimates. The algorithm is considered to converge when all parameter estimates become stable and no further improvements can be made to the likelihood value. However, to reduce computational time, it is often common practice for the algorithm to be stopped before complete convergence using heuristic approaches. In this paper, we consider various stopping criteria and evaluate their effect on fitting Gaussian mixture models (GMMs) to patient length of stay (LOS) data. Although the GMM can be successfully fitted to positively skewed data such as LOS, the fitting procedure often requires many iterations of the EM algorithm. To our knowledge, no previous study has evaluated the effect of different stopping criteria on fitting GMMs to skewed distributions. Hence, the aim of this paper is to evaluate the effect of various stopping criteria in order to select and justify their use within a patient spell classification methodology. Results illustrate that criteria based on the difference in the likelihood value and on the GMM parameters may not always be a good indicator for stopping the algorithm. In fact we show that the values of the difference in the variance parameters should be used instead, as these parameters are the last to stabilise. In addition, we also specify threshold values for the other stopping criteria.

A region and gradient based active contour model and its application in boundary tracking on anal canal ultrasound images

How to use historical data of public smart card to predict user behaviour attracts a lot of attention. This paper aims at modelling travel patterns and predicting future travel behaviour of metro system smart card holders. We apply Gaussian mixture model (GMM) on time series to model user behaviour. We propose a new method based on the perplexity for finite GMM and use expectation-maximisation (EM) algorithm to estimate parameters of GMM. In order to predict the future travel behaviour, we introduce the Gaussian process regression (GPR) to define distributions over GMM, which can not only tell the probability of travelling at a certain moment but also tell the reliability of the prediction. Experimental results show that our whole system in the centre of GMM and GPR can effectively mine the hidden knowledge of historical data of smart card, and thus model the travel patterns and predict future travel behaviour.

A novel method for machine performance degradation assessment based on fixed cycle features test