Bayesian nonparametric priors for hidden Markov random fields

Given the complexity of real-life event logs, several trace clustering techniques have been proposed to partition an event log into subsets with a lower degree of variation. In general, these techniques assume that the number of clusters is known in advance. However, this will rarely be the case in practice. Therefore, this paper presents approaches to determine the appropriate number of clusters in a trace clustering context. In order to fulfil the objective of identifying the most appropriate number of trace clusters, two approaches built on similarity are proposed: a stability- and a separation-based method. The stability-based method iteratively calculates the similarity between clustered versions of perturbed and unperturbed event logs. Alternatively, an approach based on between-cluster dissimilarity, or separation, is proposed. Regarding practical validation, both approaches are tested on multiple real-life datasets to investigate the complementarity of the different components. Our results suggest that both methods are successful in identifying an appropriate number of trace clusters.

Determining number of clusters and prototype locations via multi-scale clustering

Within the scope of cluster analysis of variables, the selection of the appropriate number of clusters is of paramount interest. The strategy of determination of the appropriate number of clusters adopted herein is based on a hypothesis testing approach. It consists in testing whether the variation of a partition quality criterion between two consecutive partitions is far removed from the expected variation under the null-hypothesis stipulating a lack of structure. Three hypothesis testing strategies are detailed and compared in the scope of clustering of variables: Gap, Weighted Gap and a statistic associated with CLV methodology. Finally, an illustration is presented based on data from a preference study.

Clustering is one of the relevant data mining tasks, which aims to process data sets in an effective way. This paper introduces a new clustering heuristic combining the E-transitive heuristic adapted to quantitative data and the k-means algorithm with the goal of ensuring the optimal number of clusters and the suitable initial cluster centres for k-means. The suggested heuris-tic, called PFK-means, is a parameter-free clustering algorithm since it does not require the prior initialization of the number of clusters. Thus, it generates progressively the initial cluster centres until the appropriate number of clusters is automatically detected. Moreover, this paper exposes a thorough comparison between the PFK-means heuristic, its diverse variants, the E-Transitive heuristic for clustering quantitative data and the traditional k-means in terms of the sum of squared errors and accuracy using different data sets. The experiments results reveal that, in general, the proposed heuristic and its variants provide the appropriate number of clusters for different real-world data sets and give good clusters quality related to the traditional k-means. Furthermore, the experiments conducted on synthetic data sets report the performance of this heuristic in terms of processing time.

The study of good governance over the past two decades has been at the forefront of the world's scientific and international circles, especially in developing countries. The purpose of this study is to evaluate the status of countries based on good governance and determine the ranking of Iran among other countries, using clustering techniques. In order to select an appropriate clustering method and the appropriate number of clusters, the Silhouette coefficient for each K-Means, Two Step and Kohonen method was calculated for the number of different clusters of each. Based on an assessment of these findings, the K-Means 12 cluster method was selected. The research findings show that, among the four countries of the world having common clusters with Iran in terms of the level of human development, Iran ranks among the 'Lowlevel developing countries' and 'Moderate developing countries'. This issue highlights the importance of improving the level of good governance indicators in the country in order to achieve a higher level of development.

In this paper, a new multi-objective (MO) fuzzy-clustering technique is developed for unsupervised land-cover classification in remote-sensing images. This technique automatically partitions the pixels in the spectral domain into several clusters. The new MO clustering algorithm uses the newly developed simulated-annealing-based multi-objective optimization technique (AMOSA) as the underlying optimization strategy. Centre-based cluster encoding is used. Each cluster is divided into several small hyperspherical sub-clusters, and the centres of all these small sub-clusters are encoded in a string to represent the whole cluster. For membership-value computation of points to different clusters, these sub-clusters are considered individually. However, for the purpose of objective-function evaluation, these sub-clusters are merged appropriately to form some variable number of whole clusters. Three objective functions, one reflecting the total compactness of the partitioning based on the Euclidean distance, one reflecting the total point-symmetrical compactness of the obtained partitioning and one reflecting the total line-symmetrical compactness of the obtained partitioning are considered here. These are optimized simultaneously using AMOSA in order to detect the appropriate number of clusters and the appropriate partitioning from remote-sensing image datasets. The point symmetry present in a particular partitioning is measured using a newly developed point-symmetry-based distance, whereas the line symmetry present in a particular partitioning is measured using a newly developed line-symmetry-based distance. Since AMOSA, as well as any other MO optimization technique, provides a set of Pareto-optimal solutions, a new method is also developed to determine a single solution from this set. Different land-cover regions in remote-sensing imagery have been classified using the proposed technique to establish its efficiency. Results are compared with those obtained by fuzzy C-means (FCM) clustering technique and a recently developed symmetry-based automatic clustering technique, the variable string length genetic point symmetry (VGAPS) technique, both qualitatively and quantitatively. Qualitative comparisons are also carried out with the segmentation results obtained by the mean-shift-based segmentation technique.

Clustering in transaction databases can find potentially useful patterns to gain some insight into the structure of the data, which can help for effective decision-making. However, one of the critical tasks in clustering is to identify the appropriate number of clusters, which will determine the performance of any process further applied to the transaction database. This paper presents a methodology to discover the optimal structure of purchase transaction data using the Davies-Bouldin and Calinski-Harabasz validity indices to obtain the number of clusters and formed them with the farthest-first traversals algorithm. The quality of the structures previously formed is evaluated with data complexity measures such as F1, F2, F3, N1 and IR. In this work, we use the support vector machine and multi-layer perceptron classification algorithms, to determine recognition ability in classification problems of more than two classes, and in the context of separability and imbalance of classes present in the groups previously obtained. The experimental results exhibit the viability of the proposed methodology for decision-making.

Clustering is a fundamental and widely used method for grouping similar records in one cluster and dissimilar records in the different cluster. In cluster analysis, a major problem is to determine the appropriate number of cluster in advance. It is difficult for a user (data miner) to estimate the appropriate number of clusters in advance. Another limitation of a well-known clustering technique called K-means is that it gets stuck at local optima. In order to overcome these limitations Genetic Algorithm (GA) based clustering techniques have been proposed in the 1990s. Since then many researchers have developed several evolutionary algorithm based clustering techniques, including GA and applied in various fields. This paper presents an up-to-date review of some major GA-based clustering techniques for the last twenty (20) years. A total of 45 ranked (i.e. based on citation reports and JCR/CORE rank) GA-based clustering approaches are reviewed, which are uses for real-life applications such as real-life data sets, highway construction projects, a Gas Company, cellular networks and satellite image segmentations. Almost two third of the techniques do not require any user to define the number of clusters. Finally, a thorough discussion and emerging research directions are presented.

We propose a new principal component analysis (PCA) for interval-valued data by using a covariance involving a fuzzy classification structure based on dissimilarity in higher dimensional space in which objects exist. The covariance for interval-valued data is obtained adaptively by evaluating the validity of the fuzzy classification structure based on the selection of an appropriate number of clusters. In order to select an appropriate number of clusters, we propose an alignment criterion to evaluate the obtained classification structure and prove the concentration of the criterion around the expected value with respect to variation of similarity among clusters. The merit of this PCA is to consider not only the projection of objects to a lower dimensional space, but also the dissimilarity of objects in a higher dimensional space by using a weighted covariance matrix. The weight is estimated as the degree of contribution for the fuzzy classification structure based on dissimilarity of objects in the higher dimensional space. A numerical example of interval-valued data consisting of human based subjective decisions shows a better performance when compared with a result of an ordinary PCA.

Although clustering is an unsupervised learning approach, most clustering algorithms require setting several parameters (such as the number of clusters, minimum density or distance threshold) in advance to work properly. In this paper, we eliminate the necessity of setting the minimum cluster size parameter of the Randomized Gravitational Clustering algorithm proposed by Gomez et al. Basically, the minimum cluster size is estimated using a heuristic that takes in consideration the functional relation between the number of clusters and the clusters with at least a given number of points. Then a data point’s region of action (region of the space assigned to a point) is defined and a cluster refinement process is proposed in order to merge overlapping clusters. Our experimental results show that the proposed algorithm is able to deal with noise, while finding an appropriate number of clusters without requiring a manual setting of the minimum cluster size.

The increasing popularity of Japanese-origin animation industries or so-called “anime” attracts more interest from already-known fans and ordinary people who are just interested in watching. However, many viewers need advice in the form of recommendations for their preferred anime. This research aims to help viewers by developing a system that could provide some recommendations for several anime series related to the current series watched by the viewers. On the other side, this research could provide a reference to other researchers, especially those whose research focuses on Machine Learning, Artificial Intelligence, and Japanese Animation culture. In this paper, the K-Means Clustering method is used to build the clustering model based on the data series, and the Elbow Method is used to determine the appropriate number of clusters. The result of this research indicates that the system can provide several titles of anime series related to the initial title of the anime series entered by the user at each iteration.

This paper presents a novel multilook SAR image segmentation algorithm with an unknown number of clusters. Firstly, the marginal probability distribution for a given SAR image is defined by a Gamma mixture model (GaMM), in which the number of components corresponds to the number of homogeneous regions needed to segment and the spatial relationship among neighboring pixels is characterized by a Markov Random Field (MRF) defined by the weighting coefficients of components in GaMM. During the algorithm iteration procedure, the number of clusters is gradually reduced by merging two components until they are equal to one. For each fixed number of clusters, the parameters of GaMM are estimated and the optimal segmentation result corresponding to the number is obtained by maximizing the marginal probability. Finally, the number of clusters with minimum global energy defined as the negative logarithm of marginal probability is indicated as the expected number of clusters with the homogeneous regions needed to be segmented, and the corresponding segmentation result is considered as the final optimal one. The experimental results from the proposed and comparing algorithms for simulated and real multilook SAR images show that the proposed algorithm can find the real number of clusters and obtain more accurate segmentation results simultaneously.

Hidden Markov random field (HMRF) models are widely used for image segmentation, as they appear naturally in problems where a spatially constrained clustering scheme is asked for. A major limitation of HMRF models concerns the automatic selection of the proper number of their states, i.e., the number of region clusters derived by the image segmentation procedure. Existing methods, including likelihood- or entropy-based criteria, and reversible Markov chain Monte Carlo methods, usually tend to yield noisy model size estimates while imposing heavy computational requirements. Recently, Dirichlet process (DP, infinite) mixture models have emerged in the cornerstone of nonparametric Bayesian statistics as promising candidates for clustering applications where the number of clusters is unknown a priori; infinite mixture models based on the original DP or spatially constrained variants of it have been applied in unsupervised image segmentation applications showing promising results. Under this motivation, to resolve the aforementioned issues of HMRF models, in this paper, we introduce a nonparametric Bayesian formulation for the HMRF model, the infinite HMRF model, formulated on the basis of a joint Dirichlet process mixture (DPM) and Markov random field (MRF) construction. We derive an efficient variational Bayesian inference algorithm for the proposed model, and we experimentally demonstrate its advantages over competing methodologies.

Bayesian nonparametric (BNP) models have recently become popular due to their flexibility in identifying the unknown number of clusters. However, they have difficulties handling heterogeneous data from multiple sources. Existing BNP methods either treat each of these sources independently - hence do not get benefits from the correlating information between them, or require to explicitly specify data sources as primary and context channels. In this paper, we present a BNP framework, termed MCNC, which has the ability to (1) discover co-patterns from multiple sources; (2) explore multi-channel data simultaneously and treat them equally; (3) automatically identify a suitable number of patterns from data; and (4) handle missing data. The key idea is to utilize a richer base measure of a BNP model being a product-space. We demonstrate our framework on synthetic and real-world datasets to discover the identity-location-time (a.k.a who-where-when) patterns. The experimental results highlight the effectiveness of our MCNC framework in both cases of complete and missing data.

Selection of initial points, the number of clusters and finding proper clusters centers are still the main challenge in clustering processes. In this paper, we suggest genetic algorithm based method which searches several solution spaces simultaneously. The solution spaces are population groups consisting of elements with similar structure. Elements in a group have the same size, while elements in different groups are of different sizes. The proposed algorithm processes the population in groups of chromosomes with one gene, two genes to k genes. These genes hold corresponding information about the cluster centers. In the proposed method, the crossover and mutation operators can accept parents with different sizes; this can lead to versatility in population and information transfer among sub-populations. We implemented the proposed method and evaluated its performance against some random datasets and the Ruspini dataset as well. The experimental results show that the proposed method could effectively determine the appropriate number of clusters and recognize their centers. Overall this research implies that using heterogeneous population in the genetic algorithm can lead to better results.