GoT: a Growing Tree Model for Clustering Ensemble

Ensemble clustering has recently emerged as a powerful tool for aggregating multiple clustering results into a probably better and more robust clustering result. While many ensemble clustering techniques have been developed in recent years, there is still a potential drawback in most of the existing algorithms. They generally neglect the possible noise inside the ensemble of base clusterings, which may exhibit a negative influence on the robustness of the consensus result. To address this, this paper proposes a new ensemble clustering algorithm based on noise-aware graph decomposition. The co-association matrix is first constructed to accumulate information from multiple base clusterings, and then sparsified by means of K-nearest neighbors. By treating the sparsified co-association matrix as the similarity matrix, the corresponding graph is further decomposed into two parts, i.e., the corruption(noise) graph and the good(denoised) graph. In particular, the denoised graph as well as its spectral embedding can be learned simultaneously by solving a trace minimization problem. Finally, upon the learned spectral embedding, the k-means discretization is performed to obtain the final consensus clustering. Experiments on multiple real-world datasets have shown the superiority of the proposed algorithm.

Weighted adaptively ensemble clustering method based on fuzzy Co-association matrix

Ensemble clustering based on evidence extracted from the co-association matrix

Spectral ensemble clustering with doubly stochastic co-association matrix

In this paper, a new method for enhancing the performance of Neural Network ensemble is proposed. The main idea of this method is creating diversity for training artificial neural networks (ANNs) using an interesting method which applies clustering ensemble and genetic algorithm. In combinational classifier systems, the more diversity in results of the base classifiers yields to better final performance. Inspiring from the boosting, the diversity of the base classifiers is provided by different train sets for base classifiers. The different train sets are derived from the original train set by adding some of data samples in train set. Finding near optimal sets is implemented using clustering ensemble technique and genetic algorithm. Finally, the majority vote fuses the outputs of trained MLPs on the new train sets from population of the last generation of GA. Experimental results demonstrate the strength of proposed method on three different datasets.

Image segmentation is a classical problem in the area of image processing, multimedia, medical image, and so on. Although there exist a lot of approaches to perform image segmentation, few of them study the image segmentation by the cluster ensemble approach. In this paper, we propose a new algorithm called the cluster ensemble algorithm (CEA) for image segmentation. Specifically, CEA first obtains two set of segmented regions which are partitioned by EM according to the color feature and the texture feature respectively. Then, it integrates these regions to k segmented regions based on the similarity measure and the fuzzy membership function. Finally, CEA performs the denoise algorithm on the segmented regions to remove the noise. The experiments show that CEA works well during the process of image segmentation.KeywordsImage SegmentationTexture FeatureCore RegionColor FeatureExpectation Maximization AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Clustering ensembles combine multiple partitions of data into a single clustering solution. It is an effective technique for improving the quality of clustering results. Current clustering ensemble algorithms are usually built on the pairwise agreements between clusterings that focus on the similarity via consensus functions, between data objects that induce similarity measures from partitions and re-cluster objects, and between clusters that collapse groups of clusters into meta-clusters. In most of those models, there is a strong assumption on IIDness (i.e., independent and identical distribution), which states that base clusterings perform independently of one another and all objects are also independent. In the real world, however, objects are generally likely related to each other through features that are either explicit or even implicit. There is also latent but definite relationship among intermediate base clusterings because they are derived from the same set of data. All these demand a further investigation of clustering ensembles that explores the interdependence characteristics of data. To solve this problem, a new coupled clustering ensemble (CCE) framework that works on the interdependence nature of objects and intermediate base clusterings is proposed in this article. The main idea is to model the coupling relationship between objects by aggregating the similarity of base clusterings, and the interactive relationship among objects by addressing their neighborhood domains. Once these interdependence relationships are discovered, they will act as critical supplements to clustering ensembles. We verified our proposed framework by using three types of consensus function: clustering-based, object-based, and cluster-based. Substantial experiments on multiple synthetic and real-life benchmark datasets indicate thatCCEcan effectively capture the implicit interdependence relationships among base clusterings and among objects with higher clustering accuracy, stability, and robustness compared to 14 state-of-the-art techniques, supported by statistical analysis. In addition, we show that the final clustering quality is dependent on the data characteristics (e.g., quality and consistency) of base clusterings in terms of sensitivity analysis. Finally, the applications in document clustering, as well as on the datasets with much larger size and dimensionality, further demonstrate the effectiveness, efficiency, and scalability of our proposed models.

Clustering ensemble technique has been shown to be effective in improving the accuracy and stability of single clustering algorithms. With the development of information technology, the amount of data, such as image, text and video, has increased rapidly. Efficiently clustering these large-scale datasets is a challenge. Clustering ensembles usually transform clustering results to a co-association matrix, and then to a graph-partition problem. These methods may suffer from information loss when computing the similarity among samples or base clusterings. Rich information between samples and base clusterings is ignored. Moreover, the results are not discrete. They need post-processing steps to obtain the final clustering result, which will deviate greatly from the real clustering result. To address this problem, we propose a co-clustering ensemble based on bilateral k-means (CEBKM) algorithm. Our algorithm can simultaneously cluster samples and base clusterings of a dataset, to fully exploit the potential information between the samples and the base clusterings. In addition, it can directly obtain the final clustering results without using other clustering algorithms. The proposed method, outperformed several state-of-the-art clustering ensemble methods in experiments conducted on real-world and toy datasets.

Region-of-interest extraction in low depth of field images using ensemble clustering and difference of Gaussian approaches

The classic Fuzzy C-means (FCM) algorithm has limited clustering performance and is prone to misclassification of border points. This study offers a bi-directional FCM clustering ensemble approach that takes local information into account (LI_BIFCM) to overcome these challenges and increase clustering quality. First, various membership matrices are created after running FCM multiple times, based on the randomization of the initial cluster centers, and a vertical ensemble is performed using the maximum membership principle. Second, after each execution of FCM, multiple local membership matrices of the sample points are created using multiple K-nearest neighbors, and a horizontal ensemble is performed. Multiple horizontal ensembles can be created using multiple FCM clustering. Finally, the final clustering results are obtained by combining the vertical and horizontal clustering ensembles. Twelve data sets were chosen for testing from both synthetic and real data sources. The LI_BIFCM clustering performance outperformed four traditional clustering algorithms and three clustering ensemble algorithms in the experiments. Furthermore, the final clustering results has a weak correlation with the bi-directional cluster ensemble parameters, indicating that the suggested technique is robust.

Data clustering is an important task and has found applications in numerous real-world problems. Since no single clustering algorithm is able to identify all different types of cluster shapes and structures, ensemble clustering was proposed to combine different partitions of the same data generated by multiple clustering algorithms. The key idea of most ensemble clustering algorithms is to find a partition that is consistent with most of the available partitions of the input data. One problem with these algorithms is their inability to handle uncertain data pairs, i.e. data pairs for which about half of the partitions put them into the same cluster and the other half do the opposite. When the number of uncertain data pairs is large, they can mislead the ensemble clustering algorithm in generating the final partition. To overcome this limitation, we propose an ensemble clustering approach based on the technique of matrix completion. The proposed algorithm constructs a partially observed similarity matrix based on the data pairs whose cluster memberships are agreed upon by most of the clustering algorithms in the ensemble. It then deploys the matrix completion algorithm to complete the similarity matrix. The final data partition is computed by applying an efficient spectral clustering algorithm to the completed matrix. Our empirical studies with multiple real-world datasets show that the proposed algorithm performs significantly better than the state-of-the-art algorithms for ensemble clustering.

Nonnegative matrix factorization for clustering ensemble based on dark knowledge

Moderate diversity for better cluster ensembles

The link-based cluster ensemble (LCE) method is applied to a high resolution satellite precipitation estimation (HSPE) algorithm, a modified form of the Precipitation Estimation from Remotely Sensed Imagery using an Artificial Neural Network Cloud Classification (PERSIANN-CCS) algorithm. The HSPE involves the following four steps: (1) segmentation of infrared cloud images into patches; (2) cloud patch feature extraction; (3) clustering and classification of cloud patches using cluster ensemble technique; and (4) dynamic application of brightness temperature (Tb) and rain rate relationships, derived using satellite observations. The LCE method combines multiple data partitions from different clustering in order to cluster the cloud patches. The results show that using the cluster ensemble increase the performance of rainfall estimates if compared to the HSPE algorithm using Self Organizing Map (SOM). The Heidke Skill Score (HSS) is improved 5% to 7% at medium and high level of rainfall thresholds.

Cluster ensembles are approaches to combine different clustering results to obtain a robust consensus partitioning. However, many cluster ensemble methods suffer from the problem of scalability since the extensive cost of calculating co-association matrix, which makes it hard to perform cluster ensemble on large scale datasets. In this paper, we proposed a scalable co-association cluster ensemble framework using a compressed version of co-association matrix formed by selecting representative points of origin instances. Experiments show that our method could get a comparable performance on medium size datasets to existing co-association ensemble method like CSPA or spectral clustering, and is able to handle large scale datasets.