ECAP: Ensemble Clustering using Affinity Propagation

Each clustering algorithm usually optimizes a qualification metric during its learning process. The qualification metric in traditional clustering algorithms considers all the features of under-consideration dataset equally; it means each feature participates in the clustering process equivalently. Considering that some features have more information than the others in a dataset (due to their lower information or their higher variances, etc.), we proposes a fuzzy weighted clustering algorithm. We name this new clustering algorithm, Fuzzy Weighted Locally Adaptive Clustering (FWLAC) algorithm. The proposed FWLAC algorithm is capable of handling imbalanced clustering. However, FWLAC algorithm suffers from its sensitivity to the two parameters that should be tuned manually. The performance of FWLAC algorithm is affected by well-tuning of its parameters. So the paper proposes two solutions to well-tuning of its two parameters. In the first solution, we propose a simple clustering ensemble framework to show the sensitivity of the WLAC algorithm to its manual well-tuning. Although it is not a try-and-error procedure, it is like a grid search, where we use different pairs of values for both parameters h 1 and h 2. Per each pair of values for parameters h 1 and h 2, the algorithm produces a partitioning. So after the grid search, we obtain a large number of partitionings. We break any of the partitionings into its clusters, and then they form an ensemble of clusters. Finally the consensus partitioning is extracted from them by a consensus function. The algorithm is not data dependent at all. For all datasets, we use a similar grid search and a similar set of values for the parameters h 1 and h 2. In this way we have proposed an alternative solution to parameter selection. We use a selection phase to select/remove some clusters from our ensemble of clusters to obtain an elite ensemble of clusters. To do this, a stability measure, normalized mutual information (NMI), was used to validate a cluster. The paper shows the effectiveness of the proposed clustering frameworks both theoretically and experimentally.

Spectral Clustering (SC) is an effective clustering method for its excellent performance in partitioning non-linearly distributed data. On the other hand, Ensemble Clustering (EC), a different clustering technology, can promote cluster quality by ensembling the results of base clusterings. In this work, we concentrate on an EC framework that utilizes SC as the base method. Nevertheless, SC suffers from scalability due to its high computational complexity in constructing the Laplacian graph and computing the corresponding eigendecomposition. In the past decades, many efforts have been made to it. However, SC suffers from the scalability issue in processing extensive data, especially in web-scale scenarios. Additionally, EC requires multiple clustering results as the ensemble bases, which further aggravates resource consumption. To address this issue, LiteWSEC, a simple yet efficient Lightweight Framework for Web-scale Spectral Ensemble Clustering, is proposed to cluster web-scale data with limited resource requirements. It adopts the Web-scale Spectral Clustering (WSC) as the base method, which has minimal space overhead without computing overall embedding explicitly. LiteWSEC is highly flexible in the memory requirement, which is adaptive to the available resource. It can partition web-scale data (e.g., <inline-formula><tex-math notation="LaTeX">$n $</tex-math></inline-formula> = 8,000 k) in an resource-limited host (e.g., memory is restricted to 1 GB). Experiments on real-world, large-scale, and web-scale datasets demonstrate both the efficiency and effectiveness of LiteWSEC over state-of-the-art SC and EC methods.

A robust adaptive clustering analysis method for automatic identification of clusters

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

Data clustering is an important task and has found applications in numerous real-world problems. Since no single clustering algorithm is able to identify all different types of cluster shapes and structures, ensemble clustering was proposed to combine different partitions of the same data generated by multiple clustering algorithms. The key idea of most ensemble clustering algorithms is to find a partition that is consistent with most of the available partitions of the input data. One problem with these algorithms is their inability to handle uncertain data pairs, i.e. data pairs for which about half of the partitions put them into the same cluster and the other half do the opposite. When the number of uncertain data pairs is large, they can mislead the ensemble clustering algorithm in generating the final partition. To overcome this limitation, we propose an ensemble clustering approach based on the technique of matrix completion. The proposed algorithm constructs a partially observed similarity matrix based on the data pairs whose cluster memberships are agreed upon by most of the clustering algorithms in the ensemble. It then deploys the matrix completion algorithm to complete the similarity matrix. The final data partition is computed by applying an efficient spectral clustering algorithm to the completed matrix. Our empirical studies with multiple real-world datasets show that the proposed algorithm performs significantly better than the state-of-the-art algorithms for ensemble clustering.

Performance evaluation results of evolutionary clustering algorithm star for clustering heterogeneous datasets

Simulated annealing spectral clustering algorithm for image segmentation

Clustering ensemble is an open proposition that aims to solve the limitation of a single cluster algorithm on the diversity of data structures in data partitions. It obtains a consensus model by integrating multiple clustering algorithms. Compared with a single clustering algorithm, it can process the data more accurately and stably, and has a better tolerance for the structure and diversity of the data. This paper proposes to integrate particle swarm optimization for clustering ensemble model. Particularly, the idea is to first use multiple classic clustering algorithms to cluster the original data set to obtain a preliminary clustering result set, and then search for a clustering result that is most consistent with the preliminary results by a particle swarm optimization procedure. In the experiments, we make use of public data sets and micro-precision to evaluate the effectiveness of the proposed clustering ensemble model, and compare it with both base clustering algorithms and other common clustering ensemble algorithms. The evaluation results show that this model can significantly improve the base clustering algorithms and outperforms common clustering ensemble algorithms in most cases.

K-means clustering is widely used due to its fast convergence, but it is sensitive to the initial condition.Therefore, many methods of initializing K-means clustering have been proposed in the literatures. Compared with Kmeans clustering, a novel clustering algorithm called affinity propagation (AP clustering) has been developed by Frey and Dueck, which can produce a good set of cluster exemplars with fast speed. Taking the convergence property of K-means and the good performance of affinity propagation, we presented a new clustering strategy which can produce much lower squared error than AP and standard K-means: initializing K-means clustering using cluster exemplars produced by AP. Numerical experiments indicated that such combined method outperforms not only AP and original K-means clustering, but also Kmeans clustering with sophisticated initial conditions designed by various methods.

In the field of mechanical fault diagnosis, in order to improve the diagnosis accuracy with high feature dimension, a method of features linear fusion method based on AP(Affinity propagation clustering) and AASC(Affinity Aggregation for Spectral Clustering) algorithm was proposed. Low dimension space was obtained by AP clustering, then using AASC algorithm to fuse this selected features, and gained a new feature, meanwhile which can be used independently as a new feature. In the phase of experiment, this method was compared with respectively LPP and PCA, it’s result indicates that this method is excellent in improving diagnosis accuracy.

Cluster ensembles are approaches to combine different clustering results to obtain a robust consensus partitioning. However, many cluster ensemble methods suffer from the problem of scalability since the extensive cost of calculating co-association matrix, which makes it hard to perform cluster ensemble on large scale datasets. In this paper, we proposed a scalable co-association cluster ensemble framework using a compressed version of co-association matrix formed by selecting representative points of origin instances. Experiments show that our method could get a comparable performance on medium size datasets to existing co-association ensemble method like CSPA or spectral clustering, and is able to handle large scale datasets.

This paper proposed StrAP (Streaming AP), extending Affinity Propagation (AP) to data steaming. AP, a new clustering algorithm, extracts the data items, or exemplars, that best represent the dataset using a message passing method. Several steps are made to build StrAP. The first one (Weighted AP) extends AP to weighted items with no loss of generality. The second one (Hierarchical WAP) is concerned with reducing the quadratic AP complexity, by applying AP on data subsets and further applying Weighted AP on the exemplars extracted from all subsets. Finally StrAP extends Hierarchical WAP to deal with changes in the data distribution. Experiments on artificial datasets, on the Intrusion Detection benchmark (KDD99) and on a real-world problem, clustering the stream of jobs submitted to the EGEE grid system, provide a comparative validation of the approach.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2007 SIAM International Conference on Data Mining (SDM)Incremental Spectral Clustering With Application to Monitoring of Evolving Blog CommunitiesHuazhong Ning, Wei Xu, Yun Chi, Yihong Gong, and Thomas HuangHuazhong Ning, Wei Xu, Yun Chi, Yihong Gong, and Thomas Huangpp.261 - 272Chapter DOI:https://doi.org/10.1137/1.9781611972771.24PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract In recent years, spectral clustering method has gained attentions because of its superior performance compared to other traditional clustering algorithms such as K-means algorithm. The existing spectral clustering algorithms are all off-line algorithms, i.e., they can not incrementally update the clustering result given a small change of the data set. However, the capability of incrementally updating is essential to some applications such as real time monitoring of the evolving communities of websphere or blogsphere. Unlike traditional stream data, these applications require incremental algorithms to handle not only insertion/deletion of data points but also similarity changes between existing items. This paper extends the standard spectral clustering to such evolving data by introducing the incidence vector/matrix to represent two kinds of dynamics in the same framework and by incrementally updating the eigenvalue system. Our incremental algorithm, initialized by a standard spectral clustering, continuously and efficiently updates the eigenvalue system and generates instant cluster labels, as the data set is evolving. The algorithm is applied to a blog data set. Compared with recomputation of the solution by standard spectral clustering, it achieves similar accuracy but with much lower computational cost. Close inspection into the blog content shows that the incremental approach can discover not only the stable blog communities but also the evolution of the individual multi-topic blogs. Previous chapter Next chapter RelatedDetails Published:2007ISBN:978-0-89871-630-6eISBN:978-1-61197-277-1 https://doi.org/10.1137/1.9781611972771Book Series Name:ProceedingsBook Code:PR127Book Pages:xiv + 648Key words:Incremental clustering, Spectral Clustering, Incidence Vector/Matrix, Web-blogs

The classic Fuzzy C-means (FCM) algorithm has limited clustering performance and is prone to misclassification of border points. This study offers a bi-directional FCM clustering ensemble approach that takes local information into account (LI_BIFCM) to overcome these challenges and increase clustering quality. First, various membership matrices are created after running FCM multiple times, based on the randomization of the initial cluster centers, and a vertical ensemble is performed using the maximum membership principle. Second, after each execution of FCM, multiple local membership matrices of the sample points are created using multiple K-nearest neighbors, and a horizontal ensemble is performed. Multiple horizontal ensembles can be created using multiple FCM clustering. Finally, the final clustering results are obtained by combining the vertical and horizontal clustering ensembles. Twelve data sets were chosen for testing from both synthetic and real data sources. The LI_BIFCM clustering performance outperformed four traditional clustering algorithms and three clustering ensemble algorithms in the experiments. Furthermore, the final clustering results has a weak correlation with the bi-directional cluster ensemble parameters, indicating that the suggested technique is robust.

An automatic affinity propagation clustering based on improved equilibrium optimizer and t-SNE for high-dimensional data