Gaussian mixture model: An application to parameter estimation and medical image classification

This paper compares performances between Gaussian Mixture Model (GMM) classifier and polynomial classifier for text-independent speaker identification. The MFCC feature set has been used for this comparison. Experimental evaluation was conducted on the POLYCOST database with 130 speakers. The importance of the prior in the polynomial classifier has been discussed in detail. Results reveal that, the identification accuracy of the polynomial classifier strongly depends on the choice of prior. For proper prior selection the polynomial classifier can perform better than the GMM classifier.KeywordsSpeaker identificationGMM classifierPolynomial classifier

One effective iterative method for estimating parameters in latent variable probabilistic models is the Expectation-Maximization (EM) algorithm, which finds extensive use in classification issues. A versatile probabilistic framework for modeling data from multiple Gaussian distributions, Gaussian Mixture Models (GMMs) employ EM for parameter estimation, which is explored in this article. Problems with class membership estimation (including means, covariances, and mixing coefficients) are the primary focus of this study. Starting with initial parameter estimations, estimating posterior probabilities in the E-step, and repeatedly updating model parameters in the M-step until convergence, the technique entails applying the EM algorithm on both synthetic and real-world datasets. In order to measure performance, we compare it to K-means and other baseline classifiers using measures like classification accuracy, Adjusted Rand Index (ARI), and log-likelihood progression. When data is distributed according to a multimodal Gaussian distribution, the findings show that EM-GMM provides better parameter estimation and better classification accuracy. Initialization procedures have a substantial effect on performance, as convergence study further shows. In addition to reiterating the EM algorithm's usefulness for probabilistic classification, this paper also describes its advantages and disadvantages and offers suggestions for improving its performance in settings with high-dimensional data and noise.

In the development of the voice conversion and the emotional speech style transformation in the text-to-speech systems, it is very important to obtain feedback information about the users’ opinion on the resulting synthetic speech quality. For this reason, the evaluations of the quality of the produced synthetic speech must often be performed for comparison. The main aim of the experiments described in this paper was to find out whether the classifier based on Gaussian mixture models (GMMs) could be applied for evaluation of male and female resynthesized speech that had been transformed from neutral to four emotional states (joy, surprise, sadness, and anger) spoken in Czech and Slovak languages. We suppose that it is possible to combine this GMM-based statistical evaluation with the classical one in the form of listening tests or it can replace them. For verification of our working hypothesis, a simple GMM emotional speech classifier with a one-level structure was realized. The next task of the performed experiment was to investigate the influence of different types and values (mean, median, standard deviation, relative maximum, etc.) of the used speech features (spectral and/or supra-segmental) on the GMM classification accuracy. The obtained GMM evaluation scores are compared with the results of the conventional listening tests based on the mean opinion scores. In addition, correctness of the GMM classification is analyzed with respect to the influence of the setting of the parameters during the GMM training—the number of mixture components and the types of speech features. The paper also describes the comparison experiment with the reference speech corpus taken from the Berlin database of emotional speech in German language as the benchmark for the evaluation of the performance of our one-level GMM classifier. The obtained results confirm practical usability of the developed GMM classifier, so we will continue in this research with the aim to increase the classification accuracy and compare it with other approaches like the support vector machines.

The expectation maximisation (EM) algorithm is an iterative maximum likelihood procedure often used for estimating the parameters of a mixture model. Theoretically, increases in the likelihood function are guaranteed as the algorithm iteratively improves upon previously derived parameter estimates. The algorithm is considered to converge when all parameter estimates become stable and no further improvements can be made to the likelihood value. However, to reduce computational time, it is often common practice for the algorithm to be stopped before complete convergence using heuristic approaches. In this paper, we consider various stopping criteria and evaluate their effect on fitting Gaussian mixture models (GMMs) to patient length of stay (LOS) data. Although the GMM can be successfully fitted to positively skewed data such as LOS, the fitting procedure often requires many iterations of the EM algorithm. To our knowledge, no previous study has evaluated the effect of different stopping criteria on fitting GMMs to skewed distributions. Hence, the aim of this paper is to evaluate the effect of various stopping criteria in order to select and justify their use within a patient spell classification methodology. Results illustrate that criteria based on the difference in the likelihood value and on the GMM parameters may not always be a good indicator for stopping the algorithm. In fact we show that the values of the difference in the variance parameters should be used instead, as these parameters are the last to stabilise. In addition, we also specify threshold values for the other stopping criteria.

The traditional Expectation-Maximization (EM) training of Gaussian Mixture Model (GMM) is essentially a batch mode procedure which requires the multiple data samples with the sufficient size to update the model parameters. This severely limits the deployment and adaptation of GMM in many real-time online systems since the newly observed data samples are expected to be incorporated into the system upon available via retraining the model. This paper presents a new online incremental EM training procedure of GMM, which aims to perform the EM training incrementally and so can adapt GMM online sample by sample. The proposed method is extended on two kinds of EM algorithms for GMM, namely, Split-and-Merge EM and the traditional EM. Experiments on both the synthetic data and a speech processing task show the advantages and efficiency of the new method.

The environment around us is very rich in acoustic information, with its scope extending beyond mere speech signals. Perception of acoustic information plays a significant role in allowing human beings to comprehend sounds that they hear in their surroundings. This paper focuses on the development of feature extraction and accurate classification of variety of acoustic sounds in unstructured environments, where adverse effects such as noise, distortion are likely to dominate. This work attempts to classify ten different unstructured real-world acoustic environments using empirical mode decomposition (EMD) which considers inherent non-stationarity of acoustic signals by decomposing the signal into intrinsic mode functions (IMFs). These IMFs are used for feature extraction. This work suggests utility of composite feature set for classification and proposes an optimized, robust, best-suitable feature set for classification of the diverse acoustic environments. For the classification task, Gaussian mixture model (GMM) and k-nearest neighbor (k-NN) classifiers are used. Utilization of this optimized best-suitable feature set yields the maximum classification accuracy of 100 % with GMM classifier and an average accuracy of 95 % for k-NN (k = 1, 3, 5). Lastly, this study presents the use and comparison of various performance metrics to evaluate classification techniques used.

Gaussian-input Gaussian mixture model for representing density maps and atomic models.

The accurate modeling of uncertain variables, such as wind power and load uncertainty, is an essential foundation for the planning operation and scheduling decisions in power systems. Currently, Gaussian mixture model (GMM) and Dirichlet process mixture model (DPMM) are the primary techniques used to characterize uncertainties in power systems, which are commonly solved by expectation-maximization (EM) algorithm. However, for the massive data of uncertain variables, the algorithm encounters challenges in accurately obtaining GMM and DPMM with a lower time consumption. To address this issue, we propose a method for GMM uncertainty modeling in power systems considering the mutual assistance of latent variables. Specifically, the GMM of uncertain variables is first constructed, and the conditional probability is employed to characterize the mutual assistance of latent variables. Then, an improved EM algorithm is developed to obtain the optimal GMM parameters, in which the expectation step (E-step) and maximization step (M-step) of the algorithm are revised using the conditional probability. Importantly, the closed-form solutions for GMM parameters are rederived in the revised E-step and M-step. Finally, the proposed uncertainty modeling method is compared with the traditional GMM and DPMM on actual wind power and load data from Australia. The proposed method performs the efficiency and accuracy in characterizing uncertainty.

This article proposes the distribution similarity measure–based Gaussian mixtures model for the contact-state (CS) modelling in force-guided robotic assembly processes of flexible rubber parts. The wrench (Cartesian force and torque) signals of the manipulated object are captured for different states of the given assembly process. The distribution similarity measure–based Gaussian mixtures model CS modelling scheme is employed in modelling the captured wrench signals for different CSs. The proposed distribution similarity measure–based Gaussian mixtures model CS modelling scheme uses the Gaussian mixtures model in modelling the captured signals. The parameters of the Gaussian mixtures models are computed using expectation maximisation. The optimal number of Gaussian mixtures model components for each CS model is determined by considering the classification success rate as an index for the similarity measure between the distribution of the captured signals and the developed models. The optimal number of Gaussian mixtures model components corresponds to the highest classification success rate; hence, object elasticity variation would be accommodated by properly choosing the optimal number of Gaussian mixtures model components. The performance of the proposed distribution similarity measure–based Gaussian mixtures model CS modelling strategy is evaluated by a test stand composed of a KUKA lightweight robot doing peg-in-hole assembly processes for flexible rubber objects. Two rubber objects with different elasticity are considered for two experiments; in the first experiment, an elastic peg of 30 Shore A hardness is considered and that of the second experiment has hardness of 6 Shore A which is even softer than the one used in experiment 1. Employing the proposed distribution similarity measure–based Gaussian mixtures model CS modelling strategy excellent classification success rate was obtained for both experiments. However, more Gaussian mixtures model components are required for the softer one that gives a strong impression of the non-stationarity behaviour increment for softer materials. Comparison is performed with the available CS modelling schemes and the distribution similarity measure–based Gaussian mixtures model is shown to provide the best classification success rate performance with a reduced computational time.

In this study, a hybrid scheme that combines Gaussian mixture model (GMM) and the k-means approach, called GMM-kmeans, is proposed for automatic audio segmentation (AAS) of popular music. Generally, the structure of a popular music is composed of verse, chorus and non-repetitive (such as intro, bridge and outro) segments. The combined GMM-kmeans scheme including mainly two developed algorithms, GMMAAS and SFS, will efficiently divide a song into these three parts. In GMM-kmeans, the GMM classifier is to recognize the vocal segments and then calculate the section boundary between them and non-repetitive sections first. The song with vocal segments extracted by GMM, containing only the remaining verse and chorus sections, is then analyzed by the k-means clustering algorithm where the verse section is further discriminated from the chorus section. In classification of verse and chorus by k-means, the developed switching frame search (SFS) algorithm with the devise of verse group-of-frames (Verse-GoF) and Chorus-GoF will accurately estimate the separation boundary of verse and chorus sections. Experimental results obtained from a musical data set of numerous Chinese popular songs show the superiority of both proposed GMMAAS and SFS.

Secure buildings are currently protected from unauthorized access by a variety of devices. Nowadays, there are many kinds of devices to guarantee the building security such as PIN pads, keys both conventional and electronic, identity cards, cryptographic and dual control procedures. However, these conventional devices can be stolen, forgotten, lost, guessed or impersonated with accuracy. Biometric system based on behavioral and/or physiological characteristics of person becomes popular as an alternative method to overcome the problem of conventional method. In this paper, voice-based biometric system is introduced for access control of building security. The ability to verify the identity of a person by analyzing his/her speech, or speaker verification, is an attractive and relatively unobtrusive means of providing security for admission into an important or secured place. An individualpsilas voice cannot be stolen, lost, forgotten, guessed, or impersonated with accuracy. In the proposed system, the access may be authorized simply by means of an enrolled user speaking into a microphone attached to the system. The proposed system then will decide whether to accept or reject the userpsilas identity claim or possibly to report insufficient confidence and request additional input before making the decision. Two approaches are adopted and evaluated to model the authorized persons, namely classical Gaussian mixture model (GMM) and artificial neural network (ANN). Experimental result confirms that in terms of false acceptance rate (FAR) and false rejection rate (FRR), the proposed voice-based access control with ANN model is better than that with GMM.

To model manipulation tasks, we propose a novel method for learning manipulation skills based on the degree of motion granularity. Even though manipulation tasks usually consist of a mixture of fine-grained and coarse-grained movements, to the best of our knowledge, manipulation skills have so far been modeled without considering their motion granularity. To model such a manipulation skill, Gaussian mixture models (GMMs) have been represented using several well-known techniques such as principal component analysis, k-means, Bayesian information criterion, and expectation-maximization (EM) algorithms. However, in this GMM, there is a problem in that when a mixture of fine-grained and coarse-grained movements is modeled as a GMM, fine-grained movements tend to be poorly represented. To resolve this issue, we measure a continuous degree of motion granularity for every time step of a manipulation task from a GMM. Then, we remodel the GMM by weighting a conventional k-means algorithm with motion granularity. Finally, we also estimate the parameters of the GMM by weighting the conventional EM with motion granularity. To validate our proposed method, we evaluate the GMM estimated using our proposed method by comparing it with those estimated by different GMMs in terms of inference, regression, and generalization using a robot arm that performs two daily tasks, namely decorating a very small area and passing through a narrow tunnel.

In this paper, the Gaussian mixture model (GMM) is introduced to implement channel multipath clustering. The GMM incorporates the covariance structure and the mean information of the channel multipaths, thus it can effectively reveal the similarity of the channel multipaths. First, the expectation-maximization (EM) algorithm is utilized to search for the posterior estimation of the GMM parameters. Then, the variational Bayesian (VB) algorithm is employed to optimize the GMM parameters to enhance the searching ability of EM and further to determine the optimal number of Gaussian distributions without resorting to cross-validation. Finally, a compact index (CI) is proposed to validate the clustering results reasonably. Thanks to the proposed CI, it is possible to find a close relationship among the GMM clustering mechanism, the multipath propagation characteristics and the CI evaluation index. Experiments with synthetic data and outdoor-to-indoor (O2I) channel data are presented to demonstrate the effectiveness of the proposed method.

This paper addresses the problem of initialization of the expectation-maximization (EM) algorithm for maximum likelihood estimation of Gaussian mixture models. In order to avoid local maxima of the likelihood function, a genetic algorithm (GA) which searches for best initial conditions of the EM algorithm is proposed. In the GA, a chromosome represents a set of initial conditions, in which initial mean vectors of mixture components are feature vectors chosen from the training set. The chromosome also encodes variances of initial spherical covariance matrices of mixture components. To evaluate each chromosome in the GA we run the EM algorithm until convergence and use the obtained log likelihood as the fitness. In computational experiments our approach was applied to clustering problem and tested on two datasets from the image processing domain. The results indicate that our method outperforms the standard multiple restart EM algorithm and is at least comparable to the state-of-the art random swap EM method.

In a distribution power network, the load model has no certain pattern or predicted behaviour due to large range of data and changes in energy consumption for end-user consumers. Thus, a powerful analysis based on probabilistic structure is required. For this paper Gaussian Mixture Model (GMM) has been used. GMM is a powerful probability model that allows different types of load distributions to be presented as a combination of several Gaussian distributions. The parameters of GMM is unknown for large random data such as real load data and these parameters can be identified by Expectation Maximization (EM) algorithm. This paper presents a method to evaluate probabilistic load data concerning the time-evolution of any type of distribution load for any duration of time. The proposed method is explained through generated load data of 100 residential houses for duration of one year.