A Spectral Clustering Method for Large-Scale Geostatistical Datasets

Spectral clustering has recently become one of the most popular modern clustering algorithms for traditional data. However, the application of this clustering method on geostatistical data produces spatially scattered clusters, which is undesirable for many geoscience applications. In this work, we develop a spectral clustering method aimed to discover spatially contiguous and meaningful clusters in multivariate geostatistical data, in which spatial dependence plays an important role. The proposed spectral clustering method relies on a similarity measure built from a non-parametric kernel estimator of the multivariate spatial dependence structure of the data, emphasizing the spatial correlation among data locations. The capability of the proposed spectral clustering method to provide spatially contiguous and meaningful clusters is illustrated using the European Geological Surveys Geochemical database.

Spectral clustering (SC) methods have been successfully applied to many real-world applications. The success of these SC methods is largely based on the manifold assumption, namely, that two nearby data points in the high-density region of a low-dimensional data manifold have the same cluster label. However, such an assumption might not always hold on high-dimensional data. When the data do not exhibit a clear low-dimensional manifold structure (e.g., high-dimensional and sparse data), the clustering performance of SC will be degraded and become even worse than K -means clustering. In this paper, motivated by the observation that the true cluster assignment matrix for high-dimensional data can be always embedded in a linear space spanned by the data, we propose the spectral embedded clustering (SEC) framework, in which a linearity regularization is explicitly added into the objective function of SC methods. More importantly, the proposed SEC framework can naturally deal with out-of-sample data. We also present a new Laplacian matrix constructed from a local regression of each pattern and incorporate it into our SEC framework to capture both local and global discriminative information for clustering. Comprehensive experiments on eight real-world high-dimensional datasets demonstrate the effectiveness and advantages of our SEC framework over existing SC methods and K-means-based clustering methods. Our SEC framework significantly outperforms SC using the Nyström algorithm on unseen data.

Fuzzy spectral clustering with robust spatial information for image segmentation

Introduction: In a hybrid AC/DC receiving-end grid, the proportion of external power injection continues to increase. The receiving-end AC grid may develop new weak nodes in grid operation. To investigate the voltage violation problem caused by the high penetration of external power injection in multi-infeed hybrid AC/DC receiving-end grids, this study proposes an adaptive spectral clustering approach for dynamic partitioning and weak node identification in hybrid AC/DC grids. Methods: Considering the degree of coupling between the AC grid and DC grid, the indices of short-circuit ratio (SCR) and multi-infeed short-circuit ratio (MISCR) are used to describe the electrical distance. To reduce the voltage control complexity, the zonal voltage regulation capability is integrated into the improved similarity matrix. Based on the spectral clustering algorithm, an improved similarity matrix is proposed, incorporating the similarity of operational state, electrical distance, and zonal voltage regulation capability, to classify the grid into several partitions. The optimal number of partitions can be determined automatically by identifying the maximum eigengap. Finally, the node with the largest singular value entropy in each partition is identified as the weak node, allowing for the quantification of its impact on system stability under disturbances. Results: The modified IEEE 118 bus system yields results that validate the feasibility of the proposed method. Discussion: This work aims to automatically determine the partitions of the hybrid AC/DC grid and identify the weak node within each partition, considering the mutual influence of the AC and DC grids and the zonal voltage regulation capability, in order to facilitate further operation and control. Conclusion: The proposed method incorporates the SCR, MISCR, and zonal reactive power compensation capability. The SCR and MISCR are used to describe the correlation mechanism between AC and DC grids, while the zonal reactive power compensation capability is utilized to enhance the accuracy of partitioning for hybrid AC/DC grids. By incorporating the eigengap in the spectral clustering method, the weak nodes in each partition are identified for further operation and control of the hybrid grid.

Spectral graph clustering methods have been a hot topic in the field of image segmentation. However, because the computational demands are needed for Spectral graph clustering methods, it has been severely limited to apply them into large data sets, such as high resolution image. It would be too expensive or even impractical for spectral decomposition to provide the optimal approximation in dealing with large or high-dimensional datasets. A novel approach aiming at reducing the computational requirements is proposed in this paper. Our approach focuses on Nystrom method for the solution of eigen-function problems. This approach enables us to use a small number of samples to infer the overall clustering solution. Based on the proposed Nystrom sampling method, this paper presents a spectral clustering algorithm for massive data analysis, and the experiments show the method is both feasible and effective.

Spectral clustering (SC) is a large family of grouping methods that partition data using eigenvectors of an affinity matrix derived from the data. Though SC methods have been successfully applied to a large number of challenging clustering scenarios, it is noteworthy that they will fail when there are significant intersections among different clusters. In this paper, based on the analysis that SC methods are able to work well when the affinity values of the points belonging to different clusters are relatively low, we propose a new method, called spectral multi-manifold clustering (SMMC), which is able to handle intersections. In our model, the data are assumed to lie on or close to multiple smooth low-dimensional manifolds, where some data manifolds are separated but some are intersecting. Then, local geometric information of the sampled data is incorporated to construct a suitable affinity matrix. Finally, spectral method is applied to this affinity matrix to group the data. Extensive experiments on synthetic as well as real datasets demonstrate the promising performance of SMMC.

This paper describes a novel classification technique-NRBF (Normalized Radial Basis Function) neural network classifier based on spectral clustering methods. The spectral method is used in the unsupervised learning part of the NRBF neural networks. Compared with other general clustering methods used in NRBF neural networks, such as KMeans, the spectral method can avoid the local minima problem and therefore multiple restarts are not necessary to obtain a good solution. This classifier was tested with satellite multi-spectral image data of New England acquired by Landsat 7 ETM+ sensors. Classification results show that this new neural network model is more accurate and robust than the conventional RBF model. Furthermore, we analyze how the number of the hidden units affects training and testing accuracy. These results suggest that this new model may be an effective method for classification of multispectral satellite image data.

Spectral clustering has exhibited a superior performance in analyzing the cluster structure of network. However, the exponentially computational complexity limits its application in analyzing large-scale social networks. To tackle this problem, many low-rank matrix approximating algorithms are proposed, of which the NystrÖm method is an approach with proved lower approximate errors. Currently, most existing sampling techniques for NystrÖm method are designed on affinity matrices, which are time-consuming to compute by some similarity metrics. Moreover, the social networks are often built on link relations, in which there is no information to construct an affinity matrix for the approximate computing of NystrÖm method for spectral clustering except for the degrees of nodes. This paper proposes a spectral clustering algorithm for large-scale social networks via a pre-coarsening sampling based NystrÖm method. By virtue of a novel triangle-based coarsening policy, the proposed algorithm first shrinks the social network into a smaller weighted network, and then does an efficient sampling for NystrÖm method to approximate the eigen-decomposition of matrix of spectral clustering. Experimental results on real large-scale social networks demonstrate that the proposed algorithm outperforms the state-of-the-art spectral clustering algorithms, which are realized by the existing sampling techniques based NystrÖm methods.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2007 SIAM International Conference on Data Mining (SDM)Incremental Spectral Clustering With Application to Monitoring of Evolving Blog CommunitiesHuazhong Ning, Wei Xu, Yun Chi, Yihong Gong, and Thomas HuangHuazhong Ning, Wei Xu, Yun Chi, Yihong Gong, and Thomas Huangpp.261 - 272Chapter DOI:https://doi.org/10.1137/1.9781611972771.24PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract In recent years, spectral clustering method has gained attentions because of its superior performance compared to other traditional clustering algorithms such as K-means algorithm. The existing spectral clustering algorithms are all off-line algorithms, i.e., they can not incrementally update the clustering result given a small change of the data set. However, the capability of incrementally updating is essential to some applications such as real time monitoring of the evolving communities of websphere or blogsphere. Unlike traditional stream data, these applications require incremental algorithms to handle not only insertion/deletion of data points but also similarity changes between existing items. This paper extends the standard spectral clustering to such evolving data by introducing the incidence vector/matrix to represent two kinds of dynamics in the same framework and by incrementally updating the eigenvalue system. Our incremental algorithm, initialized by a standard spectral clustering, continuously and efficiently updates the eigenvalue system and generates instant cluster labels, as the data set is evolving. The algorithm is applied to a blog data set. Compared with recomputation of the solution by standard spectral clustering, it achieves similar accuracy but with much lower computational cost. Close inspection into the blog content shows that the incremental approach can discover not only the stable blog communities but also the evolution of the individual multi-topic blogs. Previous chapter Next chapter RelatedDetails Published:2007ISBN:978-0-89871-630-6eISBN:978-1-61197-277-1 https://doi.org/10.1137/1.9781611972771Book Series Name:ProceedingsBook Code:PR127Book Pages:xiv + 648Key words:Incremental clustering, Spectral Clustering, Incidence Vector/Matrix, Web-blogs

Spectral Clustering (SC) is an effective clustering method for its excellent performance in partitioning non-linearly distributed data. On the other hand, Ensemble Clustering (EC), a different clustering technology, can promote cluster quality by ensembling the results of base clusterings. In this work, we concentrate on an EC framework that utilizes SC as the base method. Nevertheless, SC suffers from scalability due to its high computational complexity in constructing the Laplacian graph and computing the corresponding eigendecomposition. In the past decades, many efforts have been made to it. However, SC suffers from the scalability issue in processing extensive data, especially in web-scale scenarios. Additionally, EC requires multiple clustering results as the ensemble bases, which further aggravates resource consumption. To address this issue, LiteWSEC, a simple yet efficient Lightweight Framework for Web-scale Spectral Ensemble Clustering, is proposed to cluster web-scale data with limited resource requirements. It adopts the Web-scale Spectral Clustering (WSC) as the base method, which has minimal space overhead without computing overall embedding explicitly. LiteWSEC is highly flexible in the memory requirement, which is adaptive to the available resource. It can partition web-scale data (e.g., <inline-formula><tex-math notation="LaTeX">$n $</tex-math></inline-formula> = 8,000 k) in an resource-limited host (e.g., memory is restricted to 1 GB). Experiments on real-world, large-scale, and web-scale datasets demonstrate both the efficiency and effectiveness of LiteWSEC over state-of-the-art SC and EC methods.

In the analysis of single-cell RNA-sequencing (scRNA-seq) data, how to effectively and accurately identify cell clusters from a large number of cell mixtures is still a challenge. Low-rank representation (LRR) method has achieved excellent results in subspace clustering. But in previous studies, most LRR-based methods usually choose the original data matrix as the dictionary. In addition, the methods based on LRR usually use spectral clustering algorithm to complete cell clustering. Therefore, there is a matching problem between the spectral clustering method and the affinity matrix, which is difficult to ensure the optimal effect of clustering. Considering the above two points, we propose the DLNLRR method to better identify the cell type. First, DLNLRR can update the dictionary during the optimization process instead of using the predefined fixed dictionary, so it can realize dictionary learning and LRR learning at the same time. Second, DLNLRR can realize subspace clustering without relying on spectral clustering algorithm, that is, we can perform clustering directly based on the low-rank matrix. Finally, we carry out a large number of experiments on real single-cell datasets and experimental results show that DLNLRR is superior to other scRNA-seq data analysis algorithms in cell type identification.

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

Spectral clustering has shown a superior performance in analyzing the cluster structure. However, the exponentially computational complexity limits its application in analyzing large-scale data. To tackle this problem, many low-rank matrix approximating algorithms are proposed, of which the Nystrom method is an approach with proved lower approximate errors. The algorithms commonly combine two powerful techniques in machine learning: spectral clustering algorithms and Nystrom methods commonly used to obtain good quality low rank approximations of large matrices. This paper proposes to analyze a scalable Nystrom-based clustering algorithm with a Minimum Sum of Squared Similarities (MSSS) sampling procedure. We provide theoretical analysis of the performance of the algorithm MSSS and demonstrate its theoretical performance in comparison to the leading spectral clustering methods that use Nystrom sampling.

Spectral clustering has become a typical and efficient clustering method used in a variety of applications. The critical step of spectral clustering is the similarity measurement, which largely determines the performance of the spectral clustering method. In this paper, we propose a novel spectral clustering algorithm based on the local similarity measure of shared neighbors. This similarity measurement exploits the local density information between data points based on the weight of the shared neighbors in a directed k $$ k $$ -nearest neighbor graph with only one parameter k $$ k $$ , that is, the number of nearest neighbors. Numerical experiments on synthetic and real-world datasets demonstrate that our proposed algorithm outperforms other existing spectral clustering algorithms in terms of the clustering performance measured via the normalized mutual information, clustering accuracy, and F $$ F $$ -measure. As an example, the proposed method can provide an improvement of 15.82% in the clustering performance for the Soybean dataset.

This paper introduces a new approach for clustering large datasets based on spectral clustering and topological unsupervised learning. Spectral clustering method needs to construct an adjacency matrix and calculate the eigen-decomposition of the corresponding Laplacian matrix [4] which are computational expensive and is not easy to apply on large-scale data sets. Contrarily, the topological learning i.e. SOM method allows a projection of the dataset in low dimensional spaces that make it easy to use for very large datasets. The prototypes matrix weighted by the neighbourhood function will be used in this work to reduce the computational time of the clustering algorithm and to add the topological information to the final clustering result. We illustrate the power of this method with several real datasets. The results show a good quality of clustering results and a higher speed.