Neuronal classification from network connectivity via adjacency spectral embedding.

There have been more than 50 type clustering algorithms developed for getting meaningful information from big datasets and grouping individuals according to their characteristics. In actual researches, it is often seen that data involves all types of variables. In this case, it is very important to select appropriate clustering algorithm according to different data types. In this study, we will provide information about EM(Expectation Maximization),Two–Step Clustering methods which are developed in recent years and one of the best methods for data sets containing mixed types of variables. And the second aim is to compare the methods by producing a data set from health field information.These algorithms are generally recommended for large data sets but there are also used n medium-sized data sets. Medium- sized data sets are more often in actual researches.Therefore, fifty people for control group and fifty people for patients that have polycystic over syndrome were taken to the study. Totally nineteen variables were measured from these subjects and thirteen of them were quantitative, six of them were qualitative.Clusters were obtained by EM and Two-Step cluster methods.To evaluate the relationships between the clusters obtained from algorithms and actually known patient, control groups were analyzed by Kappa coefficient. It was found that EM clustering algorithm has highest compliance coefficient comparing with Two-Step cluster(Kappa=0,740;p

On convergence and parameter selection of the EM and DA-EM algorithms for Gaussian mixtures

It is now widely accepted that knowledge can be acquired from networks by clustering their vertices according to the connection profiles. Many methods have been proposed and in this paper we concentrate on the Stochastic Block Model (SBM). The clustering of vertices and the estimation of SBM model parameters have been subject to previous work, and numerous inference strategies such as variational expectation maximization (EM) and classification EM have been proposed. However, SBM still suffers from a lack of criteria to estimate the number of components in the mixture. To our knowledge, only one model-based criterion, Integrated Complete-data Likelihood (ICL), has been derived for SBM in the literature. It relies on an asymptotic approximation of the integrated complete-data likelihood and recent studies have shown that it tends to be too conservative in the case of small networks. To tackle this issue, we propose a new criterion that we call Integrated Likelihood Variational Bayes (ILvb), based on a non-asymptotic approximation of the marginal likelihood. We describe how the criterion can be computed through a variational Bayes EM algorithm.

Use of a fuzzy granulation–degranulation criterion for assessing cluster validity

Several criminal profiling systems have been developed to assist the Law Enforcement Agencies in solving crimes but the techniques employed in most of the systems lack the ability to cluster criminal based on their behavioral characteristics. This paper reviewed different clustering techniques used in criminal profiling and then selects one fuzzy clustering algorithm (Expectation Maximization) and two hard clustering algorithm (K-means and Hierarchical). The selected algorithms were then developed and tested on real life data to produce "profiles" of criminal activity and behavior of criminals. The algorithms were implemented using WEKA software package. The performance of the algorithms was evaluated using cluster accuracy and time complexity. The results show that Expectation Maximization algorithm gave a 90.5% clusters accuracy in 8.5s, while K-Means had 62.6% in 0.09s and Hierarchical with 51.9% in 0.11s. In conclusion, soft clustering algorithm performs better than hard clustering algorithm in analyzing criminal data.
 Keywords: Clustering Algorithm, Profiling, Crime, Membership value

The rapidly increasing availability of online social networks and the well-known effect of social influence have motivated research on social-network based recommenders. Social influence and selection together lead to the formation of communities of like-minded and well connected users. Exploiting the clustering of users and items is one of the most important approaches for model-based recommendation. Users may belong to multiple communities or groups, but only a few clustering algorithms allow clusters to overlap. One of these algorithms is the probabilistic EM clustering method, which assumes that data is generated from a mixture of Gaussian models. The mixed membership stochastic block model (MMB) transfers the idea of EM clustering from conventional, non-relational data to social network data. In this paper, we introduce a generalized stochastic blockmodel (GSBM) that models not only the social relations but also the rating behavior. This model learns the mixed group membership assignments for both users and items in an SRN. GSBM can predict the future behavior of users, both the rating of items and creation of links to other users. We performed experiments on two real life datasets from Epinions.com and Flixster.com, demonstrating the accuracy of the proposed GSBM for rating prediction as well as link prediction.

Classical connectivity is a vital metric to assess the reliability of interconnection networks, while it has defects in the defective assumption that all neighbours of one node may fail concurrently. To overcome this deficiency, many new generalizations of traditional connectivity, such as g-component (edge) connectivity, restricted (edge) connectivity, g-extra (edge) connectivity and so on, have been suggested. For any positive integer a, the a-regular edge connectivity of a connected graph G is the minimum cardinality of an edge cut, whose deletion disconnects G so that each component is a a-regular graph. In this work, we investigate the a-regular edge connectivity of some regular networks, which provides a novel idea for further exploring the reliability of networks. Finally, simulations are carried out to compare the a-regular connectivity with other types of connectivity in some regular networks. The comparison results imply that the a-regular edge connectivity has its superiority.

Objective: Data being missed is an unavoidable problem in cohort studies. This paper compares the imputation effect of eight common missing data imputation methods involved in cutting longitudinal data through simulation study to provide a valuable reference for the treatment of missing data in longitudinal studies. Methods: The simulation study is based on R language software and generates missing longitudinal data by the Monte Carlo method. By comparing the average absolute deviation, average relative deviation, and TypeⅠerror from the regression analysis of different imputation methods, the imputation effect of varying imputation methods on missing longitudinal data and the influence on subsequent multivariate analysis are evaluated. Results: The mean imputation, k nearest neighbor (KNN), regression imputation, and random forest all have a similar imputation effect, which is also steady. However, the hot deck is inferior to the above imputation methods. K-means clustering and expectation maximization (EM) algorithm are among the worst and unstable. Mean imputation, EM algorithm, random forest, KNN, and regression imputation can control TypeⅠerror. Still, multiple imputations, hot deck, and K-means clustering cannot effectively manage the TypeⅠerror. Conclusions: For missing data in longitudinal studies, mean imputation, KNN, regression imputation, and random forest can be used as better imputation methods under the mechanism of missing at random. When the missing ratio is not too large, multiple imputations and hot deck can also perform well, but K-means clustering and EM algorithm are not recommended.

Model-based clustering approaches concern the paradigm of exploratory data analysis relying on the finite mixture model to automatically find a latent structure governing observed data. They are one of the most popular and successful approaches in cluster analysis. The mixture density estimation is generally performed by maximizing the observed-data log-likelihood by using the expectation-maximization (EM) algorithm. However, it is well-known that the EM algorithm initialization is crucial. In addition, the standard EM algorithm requires the number of clusters to be known a priori. Some solutions have been provided in [31], [12] for model-based clustering with Gaussian mixture models for multivariate data. In this paper we focus on model-based curve clustering approaches, when the data are curves rather than vectorial data, based on regression mixtures. We propose a new robust EM algorithm for clustering curves. We extend the model-based clustering approach presented in [31] for Gaussian mixture models, to the case of curve clustering by regression mixtures, including polynomial regression mixtures as well as spline or B-spline regressions mixtures. Our approach both handles the problem of initialization and the one of choosing the optimal number of clusters as the EM learning proceeds, rather than in a twofold scheme. This is achieved by optimizing a penalized log-likelihood criterion. A simulation study confirms the potential benefit of the proposed algorithm in terms of robustness regarding initialization and funding the actual number of clusters.

EM algorithm is widely used in clustering domain because of its easy implementation and small storage space. CEM algorithm, which is considered as a classification version of EM algorithm, is another common used clustering algorithm. With the development of technology, we obtain more and more data. This results in slow computation of EM and CEM algorithms. Binning data seems to be efficient in gaining computation time by reducing the number of observations to the number of bins. Thus, EM and CEM algorithms applied to binned data were proposed: binned-EM and bin-EM-CEM algorithms. Moreover, fourteen parsimonious Gaussian mixture models, generated according to eigenvalue decomposition of the variance matrices of the mixture components, have less parameters than the most general model. By applying the EM and CEM algorithms of parsimonious models, estimation process is simplified and then accelerated. In this paper, to combine the advantages of binned data and parsimonious Gaussian mixture models, we develop bin-EM-CEM algorithms of spherical parsimonious Gaussian mixture models.

Gaussian mixture has been widely used for data modeling and analysis and the EM algorithm is generally employed for its parameter learning. However, the EM algorithm may be trapped into a local maximum of the likelihood and even leads to a wrong result if the number of components is not appropriately set. Recently, the competitive EM (CEM) algorithm for Gaussian mixtures, a new kind of split-and-merge learning algorithm with certain competitive mechanism on estimated components of the EM algorithm, has been constructed to overcome these drawbacks. In this paper, we construct a new CEM algorithm through the Bayesian Ying-Yang (BYY) harmony stop criterion, instead of the previously used MML criterion. It is demonstrated by the simulation experiments that our proposed CEM algorithm outperforms the original one on both model selection and parameter estimation.KeywordsCompetitive EM (CEM) algorithmGaussian mixtureBayesian Ying-Yang (BYY) harmony learningModel selection

Clustering is a fundamental problem in statistics and machine learning, whose solution is commonly computed by the Expectation-Maximization (EM) method, which finds a locally optimal solution for an objective function called log-likelihood. Since the surface of the log-likelihood function is non convex, a stochastic search with Markov Chain Monte Carlo (MCMC) methods can help escaping locally optimal solutions. In this article, we tackle two fundamental conflicting goals: Finding higher quality solutions and achieving faster convergence. With that motivation in mind, we introduce an efficient algorithm that combines elements of the EM and MCMC methods to find clustering solutions that are qualitatively better than those found by the standard EM method. Moreover, our hybrid algorithm allows tuning model parameters and understanding the uncertainty in their estimation. The main issue with MCMC methods is that they generally require a very large number of iterations to explore the posterior of each model parameter. Convergence is accelerated by several algorithmic improvements which include sufficient statistics, simplified model parameter priors, fixing covariance matrices and iterative sampling from small blocks of the data set. A brief experimental evaluation shows promising results.

In this paper, an unsupervised image segmentation algorithm is proposed, which combines spatial constraints with a kernel fuzzy c-means (KFCM) clustering algorithm. Conventional KFCM clustering segmentation algorithm does not incorporate the spatial context information of image, which makes it sensitive to the noise and intensity variations. In order to overcome the shortcomings, the contents of image is characterized by Gaussian mixture model, and the parameters of model are estimated by modified expectation maximization (EM) algorithm, which overcomes the classical EM algorithm drawbacks that easily trap in local maxima and be susceptible to initial value. According to the maximum a posterior theorem, we can get the pixel maximum posterior probability. We redefine the objective function of the KFCM algorithm which incorporates the pixel maximum posteriori probability, by minimizing the fuzzy objective function, the fuzzy segmentation algorithm is derived. The experimental results on a synthetic image and a real magnetic resonance image show that the proposed algorithm is more effective than the conventional FCM algorithm without local spatial constraints.

In the last decades, Gaussian Mixture Models (GMMs) have attracted considerable interest in data mining and pattern recognition. A GMM-based clustering algorithm models a dataset with a mixture of multiple Gaussian components and estimates the model parameters using the Expectation-Maximization (EM) algorithm. Recently, a new Locally Consistent GMM (LCGMM) has been proposed to improve the clustering performance by exploiting the local manifold structure of the data using a p nearest neighbor graph. In addition to the underlying manifold structure, many other forms of prior knowledge may guide the clustering process and improve the performance. In this paper, we introduce a Semi-Supervised LCGMM (Semi-LCGMM), where the prior knowledge is provided in the form of class labels of partial data. In particular, the new Semi-LCGMM incorporates the prior knowledge into the maximum likelihood function of the original LCGMM, and the model parameters are estimated using the EM algorithm. It is worth noting that, in our algorithm, each class may be modeled by multiple Gaussian components while in the unsupervised setting each class is modeled by a single Gaussian component. Our algorithm has shown promising results in many different applications, including clustering breast cancer data, heart disease data, handwritten digit images, human face images, and image segmentation.

Separation of uncertainty and variability in quantitative microbial risk assessment models