Spectral Clustering Trough Topological Learning for Large Datasets

Spectral graph clustering methods have been a hot topic in the field of image segmentation. However, because the computational demands are needed for Spectral graph clustering methods, it has been severely limited to apply them into large data sets, such as high resolution image. It would be too expensive or even impractical for spectral decomposition to provide the optimal approximation in dealing with large or high-dimensional datasets. A novel approach aiming at reducing the computational requirements is proposed in this paper. Our approach focuses on Nystrom method for the solution of eigen-function problems. This approach enables us to use a small number of samples to infer the overall clustering solution. Based on the proposed Nystrom sampling method, this paper presents a spectral clustering algorithm for massive data analysis, and the experiments show the method is both feasible and effective.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2007 SIAM International Conference on Data Mining (SDM)Incremental Spectral Clustering With Application to Monitoring of Evolving Blog CommunitiesHuazhong Ning, Wei Xu, Yun Chi, Yihong Gong, and Thomas HuangHuazhong Ning, Wei Xu, Yun Chi, Yihong Gong, and Thomas Huangpp.261 - 272Chapter DOI:https://doi.org/10.1137/1.9781611972771.24PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract In recent years, spectral clustering method has gained attentions because of its superior performance compared to other traditional clustering algorithms such as K-means algorithm. The existing spectral clustering algorithms are all off-line algorithms, i.e., they can not incrementally update the clustering result given a small change of the data set. However, the capability of incrementally updating is essential to some applications such as real time monitoring of the evolving communities of websphere or blogsphere. Unlike traditional stream data, these applications require incremental algorithms to handle not only insertion/deletion of data points but also similarity changes between existing items. This paper extends the standard spectral clustering to such evolving data by introducing the incidence vector/matrix to represent two kinds of dynamics in the same framework and by incrementally updating the eigenvalue system. Our incremental algorithm, initialized by a standard spectral clustering, continuously and efficiently updates the eigenvalue system and generates instant cluster labels, as the data set is evolving. The algorithm is applied to a blog data set. Compared with recomputation of the solution by standard spectral clustering, it achieves similar accuracy but with much lower computational cost. Close inspection into the blog content shows that the incremental approach can discover not only the stable blog communities but also the evolution of the individual multi-topic blogs. Previous chapter Next chapter RelatedDetails Published:2007ISBN:978-0-89871-630-6eISBN:978-1-61197-277-1 https://doi.org/10.1137/1.9781611972771Book Series Name:ProceedingsBook Code:PR127Book Pages:xiv + 648Key words:Incremental clustering, Spectral Clustering, Incidence Vector/Matrix, Web-blogs

Spectral Clustering (SC) is an effective clustering method for its excellent performance in partitioning non-linearly distributed data. On the other hand, Ensemble Clustering (EC), a different clustering technology, can promote cluster quality by ensembling the results of base clusterings. In this work, we concentrate on an EC framework that utilizes SC as the base method. Nevertheless, SC suffers from scalability due to its high computational complexity in constructing the Laplacian graph and computing the corresponding eigendecomposition. In the past decades, many efforts have been made to it. However, SC suffers from the scalability issue in processing extensive data, especially in web-scale scenarios. Additionally, EC requires multiple clustering results as the ensemble bases, which further aggravates resource consumption. To address this issue, LiteWSEC, a simple yet efficient Lightweight Framework for Web-scale Spectral Ensemble Clustering, is proposed to cluster web-scale data with limited resource requirements. It adopts the Web-scale Spectral Clustering (WSC) as the base method, which has minimal space overhead without computing overall embedding explicitly. LiteWSEC is highly flexible in the memory requirement, which is adaptive to the available resource. It can partition web-scale data (e.g., <inline-formula><tex-math notation="LaTeX">$n $</tex-math></inline-formula> = 8,000 k) in an resource-limited host (e.g., memory is restricted to 1 GB). Experiments on real-world, large-scale, and web-scale datasets demonstrate both the efficiency and effectiveness of LiteWSEC over state-of-the-art SC and EC methods.

A robust adaptive clustering analysis method for automatic identification of clusters

BackgroundMulti-dimensional scaling (MDS) is aimed to represent high dimensional data in a low dimensional space with preservation of the similarities between data points. This reduction in dimensionality is crucial for analyzing and revealing the genuine structure hidden in the data. For noisy data, dimension reduction can effectively reduce the effect of noise on the embedded structure. For large data set, dimension reduction can effectively reduce information retrieval complexity. Thus, MDS techniques are used in many applications of data mining and gene network research. However, although there have been a number of studies that applied MDS techniques to genomics research, the number of analyzed data points was restricted by the high computational complexity of MDS. In general, a non-metric MDS method is faster than a metric MDS, but it does not preserve the true relationships. The computational complexity of most metric MDS methods is over O(N2), so that it is difficult to process a data set of a large number of genes N, such as in the case of whole genome microarray data.ResultsWe developed a new rapid metric MDS method with a low computational complexity, making metric MDS applicable for large data sets. Computer simulation showed that the new method of split-and-combine MDS (SC-MDS) is fast, accurate and efficient. Our empirical studies using microarray data on the yeast cell cycle showed that the performance of K-means in the reduced dimensional space is similar to or slightly better than that of K-means in the original space, but about three times faster to obtain the clustering results. Our clustering results using SC-MDS are more stable than those in the original space. Hence, the proposed SC-MDS is useful for analyzing whole genome data.ConclusionOur new method reduces the computational complexity from O(N3) to O(N) when the dimension of the feature space is far less than the number of genes N, and it successfully reconstructs the low dimensional representation as does the classical MDS. Its performance depends on the grouping method and the minimal number of the intersection points between groups. Feasible methods for grouping methods are suggested; each group must contain both neighboring and far apart data points. Our method can represent high dimensional large data set in a low dimensional space not only efficiently but also effectively.

Subspace clustering methods based on spectral clustering have been very popular due to their theoretical guarantees and empirical success. However, considering the constraint information of data, these subspace-clustering-based constraint clustering algorithms are difficult for the high-dimensional data with data nuisances to achieve better clustering results. This paper proposes a novel constraint spectral clustering algorithm based on the program of sparse subspace. Firstly, constraint term which are suitable for sparse subspace model are established according to the different statuses of representation matrix. Then a novel semi-supervised sparse subspace model is presented with the constraint terms mentioned above. Finally, the final clustering results could be acquired by spectral clustering under the guidance of constraint information. Experiments on two real-world dataset verify the property of the algorithm and show that this approach could achieve better clustering accuracy than others.

Use of a fuzzy granulation–degranulation criterion for assessing cluster validity

Spectral clustering (SC) methods have been successfully applied to many real-world applications. The success of these SC methods is largely based on the manifold assumption, namely, that two nearby data points in the high-density region of a low-dimensional data manifold have the same cluster label. However, such an assumption might not always hold on high-dimensional data. When the data do not exhibit a clear low-dimensional manifold structure (e.g., high-dimensional and sparse data), the clustering performance of SC will be degraded and become even worse than K -means clustering. In this paper, motivated by the observation that the true cluster assignment matrix for high-dimensional data can be always embedded in a linear space spanned by the data, we propose the spectral embedded clustering (SEC) framework, in which a linearity regularization is explicitly added into the objective function of SC methods. More importantly, the proposed SEC framework can naturally deal with out-of-sample data. We also present a new Laplacian matrix constructed from a local regression of each pattern and incorporate it into our SEC framework to capture both local and global discriminative information for clustering. Comprehensive experiments on eight real-world high-dimensional datasets demonstrate the effectiveness and advantages of our SEC framework over existing SC methods and K-means-based clustering methods. Our SEC framework significantly outperforms SC using the Nyström algorithm on unseen data.

Previous multi-view spectral clustering methods are a two-step strategy, which first learns a fixed common representation (or common affinity matrix) of all the views from original data and then conducts k-means clustering on the resulting common affinity matrix. The two-step strategy is not able to output reasonable clustering performance since the goal of the first step (i.e., the common affinity matrix learning) is not designed for achieving the optimal clustering result. Moreover, the two-step strategy learns the common affinity matrix from original data, which often contain noise and redundancy to influence the quality of the common affinity matrix. To address these issues, in this paper, we design a novel One-step Multi-view Spectral Clustering (OMSC) method to output the common affinity matrix as the final clustering result. In the proposed method, the goal of the common affinity matrix learning is designed to achieving optimal clustering result and the common affinity matrix is learned from low-dimensional data where the noise and redundancy of original high-dimensional data have been removed. We further propose an iterative optimization method to fast solve the proposed objective function. Experimental results on both synthetic datasets and public datasets validated the effectiveness of our proposed method, comparing to the state-of-the-art methods for multi-view clustering.

Spectral clustering is one of the most popular modern clustering techniques for conventional data. However, the application of the general spectral clustering method in the geostatistical data framework poses a double challenge. Firstly, applied to geostatistical data, the general spectral clustering method produces clusters that are spatially non-contiguous which is undesirable for many geoscience applications. Secondly, it is limited in its applicability to large-scale problems due to its high computational complexity. This paper presents a spectral clustering method dedicated to large-scale geostatistical datasets in which spatial dependence plays an important role. It extends a previous work to large-scale geostatistical datasets by computing the similarity matrix only at a reduced set of locations over the study domain referred to as anchor locations. It has the advantage of using all data during the computation of the similarity matrix at anchor locations; so there is no sacrifice of data. The spectral clustering algorithm can then be efficiently performed on this similarity matrix at anchor locations rather than all data locations. Given the resulting cluster labels of anchor locations, a weighted k-nearest-neighbour classifier is trained using their geographical coordinates as covariates and their cluster labels as the response. The assignment of clustering membership to the entire data locations is obtained by applying the trained classifier. The effectiveness of the proposed method to discover spatially contiguous and meaningful clusters in large-scale geostatistical datasets is illustrated using the US National Geochemical Survey database.

Spectral clustering (SC) is a large family of grouping methods that partition data using eigenvectors of an affinity matrix derived from the data. Though SC methods have been successfully applied to a large number of challenging clustering scenarios, it is noteworthy that they will fail when there are significant intersections among different clusters. In this paper, based on the analysis that SC methods are able to work well when the affinity values of the points belonging to different clusters are relatively low, we propose a new method, called spectral multi-manifold clustering (SMMC), which is able to handle intersections. In our model, the data are assumed to lie on or close to multiple smooth low-dimensional manifolds, where some data manifolds are separated but some are intersecting. Then, local geometric information of the sampled data is incorporated to construct a suitable affinity matrix. Finally, spectral method is applied to this affinity matrix to group the data. Extensive experiments on synthetic as well as real datasets demonstrate the promising performance of SMMC.

In recent years, k-way spectral clustering is one of the most popular clustering algorithms. Due to its computational complexity of $O(n^{3})$, with n the number of data points it is a challenging task to apply traditional spectral clustering algorithms for large-scale data sets. In this paper, we propose a novel algorithm called k-way Fast Spectral Clustering (KFSC) based on kd-tree. This algorithm is based on a theoretical analysis that provides a statistical characterization of the effect of local distortion on the mis-clustering rate. The computational complexity is $O(ndl)+O(m^{3})+O(m^{2})$, where l is the depth of kd-tree, m is the number of representative sets (set of similar data points) and d is the dimension of data. Extensive experimental results on real data sets and synthetic data sets demonstrate that the proposed methodology outperforms Normalized cut (Ncut) in terms of computation, while achieving comparable clustering accuracy.

In the analysis of single-cell RNA-sequencing (scRNA-seq) data, how to effectively and accurately identify cell clusters from a large number of cell mixtures is still a challenge. Low-rank representation (LRR) method has achieved excellent results in subspace clustering. But in previous studies, most LRR-based methods usually choose the original data matrix as the dictionary. In addition, the methods based on LRR usually use spectral clustering algorithm to complete cell clustering. Therefore, there is a matching problem between the spectral clustering method and the affinity matrix, which is difficult to ensure the optimal effect of clustering. Considering the above two points, we propose the DLNLRR method to better identify the cell type. First, DLNLRR can update the dictionary during the optimization process instead of using the predefined fixed dictionary, so it can realize dictionary learning and LRR learning at the same time. Second, DLNLRR can realize subspace clustering without relying on spectral clustering algorithm, that is, we can perform clustering directly based on the low-rank matrix. Finally, we carry out a large number of experiments on real single-cell datasets and experimental results show that DLNLRR is superior to other scRNA-seq data analysis algorithms in cell type identification.

In recent years, multiview clustering algorithms have achieved promising performance by exploiting the complementarity and consistency of different views. However, many multiview spectral clustering methods only focus on the consistent information of views, and the time cost of feature decomposition is expensive. Moreover, these methods also require postprocessing (e.g., k-means) to obtain the final clustering results. To overcome these limitations simultaneously, we propose a novel multiview clustering algorithm. First, the method removes the inconsistent information of the views through cross-view measurement to maintain consistent information. These inconsistencies may be caused by noise, corruptions, or view-specific properties and will affect the quality of the similarity matrix. Then, we learn a consensus embedding matrix with nonnegative constraints by performing a low-rank decomposition of the consistency information. In this way, we can replace the eigendecomposition of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> Laplacian matrix in spectral clustering with the singular value decomposition of a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> low-rank matrix to reduce the computational burden, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\ll }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> . Furthermore, due to the nonnegative constraint, we can directly obtain the clustering results. Also, to consider the diversity of views, adaptive weighting is applied to different view data. Compared to state-of-the-art multiview clustering methods on five benchmark multiview datasets, we demonstrate the superiority and effectiveness of our approach. We release the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hulu88/FAMvC</uri> .

&lt;p&gt;&lt;em&gt;The MERS-Cov virus has spread to other countries outside Saudi Arabia. This is because the MERS-CoV virus can mutate rapidly so it is feared that it could threaten public health and even world health. This virus develops and becomes an acute respiratory disease and the mortality rate reaches 30% among 536 cases. One way to classify the MERS-CoV virus is by grouping the DNA sequences of the MERS-CoV virus which have similar characteristics and functions. Spectral clustering is a grouping method that can identify DNA gene expression. This method is also able to partition DNA data with a more complex structure than the partition clustering method. The purpose of this study was to analyze the MERS-CoV virus clustering using the spectral clustering method and the k-means algorithm. This study used a quantitative descriptive literature approach. The results showed that the results of clustering using the spectral clustering method and the k-means algorithm produced three clusters and were more homogeneous than clustering using k-means only.&lt;/em&gt;&lt;/p&gt;