Flexible constrained spectral clustering

Spectral Clustering (SC) is an effective clustering method for its excellent performance in partitioning non-linearly distributed data. On the other hand, Ensemble Clustering (EC), a different clustering technology, can promote cluster quality by ensembling the results of base clusterings. In this work, we concentrate on an EC framework that utilizes SC as the base method. Nevertheless, SC suffers from scalability due to its high computational complexity in constructing the Laplacian graph and computing the corresponding eigendecomposition. In the past decades, many efforts have been made to it. However, SC suffers from the scalability issue in processing extensive data, especially in web-scale scenarios. Additionally, EC requires multiple clustering results as the ensemble bases, which further aggravates resource consumption. To address this issue, LiteWSEC, a simple yet efficient Lightweight Framework for Web-scale Spectral Ensemble Clustering, is proposed to cluster web-scale data with limited resource requirements. It adopts the Web-scale Spectral Clustering (WSC) as the base method, which has minimal space overhead without computing overall embedding explicitly. LiteWSEC is highly flexible in the memory requirement, which is adaptive to the available resource. It can partition web-scale data (e.g., <inline-formula><tex-math notation="LaTeX">$n $</tex-math></inline-formula> = 8,000 k) in an resource-limited host (e.g., memory is restricted to 1 GB). Experiments on real-world, large-scale, and web-scale datasets demonstrate both the efficiency and effectiveness of LiteWSEC over state-of-the-art SC and EC methods.

Spectral clustering is a well-known method for clustering the vertices of an undirected network. Although its use in network psychometrics has been limited, spectral clustering has a close relationship to the commonly used walktrap algorithm. In this article, we report results from simulation experiments designed to evaluate the ability of spectral clustering and the walktrap algorithm to recover underlying cluster (or community) structure in networks. The salient findings include: (a) the recovery performance of the walktrap algorithm can be improved by using K-means clustering instead of hierarchical clustering; (b) K-means and K-median clustering led to comparable recovery performance when used to cluster vertices based on the eigenvectors of Laplacian matrices in spectral clustering; (c) spectral clustering using the unnormalized Laplacian matrix generally yielded inferior cluster recovery in comparison to the other methods; (d) when the correct number of clusters was provided for the methods, spectral clustering using the normalized Laplacian matrix led to better recovery than the walktrap algorithm; and (e) when the correct number of clusters was not provided, the walktrap algorithm using the Qw modularity index was better than spectral clustering using the eigengap heuristic at determining the appropriate number of clusters. Overall, both the walktrap algorithm and spectral clustering of the normalized Laplacian matrix are effective for partitioning the vertices of undirected networks, with the latter performing better in most instances. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

In this paper, we propose a new image clustering algorithm, referred to as clustering using local discriminant models and global integration (LDMGI). To deal with the data points sampled from a nonlinear manifold, for each data point, we construct a local clique comprising this data point and its neighboring data points. Inspired by the Fisher criterion, we use a local discriminant model for each local clique to evaluate the clustering performance of samples within the local clique. To obtain the clustering result, we further propose a unified objective function to globally integrate the local models of all the local cliques. With the unified objective function, spectral relaxation and spectral rotation are used to obtain the binary cluster indicator matrix for all the samples. We show that LDMGI shares a similar objective function with the spectral clustering (SC) algorithms, e.g., normalized cut (NCut). In contrast to NCut in which the Laplacian matrix is directly calculated based upon a Gaussian function, a new Laplacian matrix is learnt in LDMGI by exploiting both manifold structure and local discriminant information. We also prove that K-means and discriminative K-means (DisKmeans) are both special cases of LDMGI. Extensive experiments on several benchmark image datasets demonstrate the effectiveness of LDMGI. We observe in the experiments that LDMGI is more robust to algorithmic parameter, when compared with NCut. Thus, LDMGI is more appealing for the real image clustering applications in which the ground truth is generally not available for tuning algorithmic parameters.

Spectral clustering (SC) is an unsupervised learning method that has been widely used in industrial product analysis. Compressive spectral clustering (CSC) effectively accelerates clustering by leveraging graph filter and random sampling techniques. However, CSC suffers from two major problems. First, the direct use of the dichotomy and eigencount techniques for estimating Laplacian matrix's k-th eigenvalue is expensive. Second, the computation of interpolation is time-consuming because it requires to repeat matrix-vector product for every cluster in each iteration. To address these problems, we propose a new method called fast compressive spectral clustering (FCSC). Our method addresses the first problem by assuming that the eigenvalues approximately satisfy local uniform distribution, and addresses the second problem by recalculating the pairwise similarity between nodes with low-dimensional representation to reconstruct denoised laplacian matrix. The time complexity of reconstruction is linear with the number of non-zeros in Laplacian matrix. As experimentally demonstrated on both artificial and real-world datasets, our method significantly reduces the computation time while preserving high clustering accuracy comparable to previous designs, demonstrating the effectiveness of FCSC.

Semi-supervised or constrained clustering refers to clustering data instances in the presence of very limited supervisory information. Although it has been widely investigated in traditional clustering algorithms such as k-means, hierarchical and spectral clustering, little research has addressed the problem of incorporating such knowledge into swarm-intelligence based clustering algorithms. In this study, we present a new Constrained Ant Clustering Algorithm (CACA) with its application to the task of aspect category identification in product reviews, a central clustering task in Aspect-Based Sentiment Analysis (ABSA). We validate our CACA on benchmark datasets, and we show its effectiveness to real-world datasets for ABSA.

Protein sequence data analysis is a fundamental task in bioinformatics, supporting the exploration of biological variations and the identification of functional relationships among proteins. This study presents a performance analysis of four clustering algorithms, which include Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Agglomerative Hierarchical Clustering, and Spectral Clustering, applied to protein sequence datasets. Feature extraction was conducted using the Discere package in Python, generating 27 numerical attributes from protein sequences. The optimal number of clusters for BIRCH, Agglomerative, and Spectral Clustering was determined using the Elbow method, while DBSCAN parameters (MinPts, Eps) were tuned using the sorted k-distance plot. Clustering performance was assessed using the Silhouette Score. Among the algorithms, DBSCAN produced the highest silhouette score of 0.8105, whereas BIRCH achieved a strong balance between clustering quality, with a score of 0.7405, and computational efficiency. Agglomerative clustering provided moderate results with a score of 0.6779, while Spectral clustering yielded the lowest score of 0.6310 but demonstrated flexibility in capturing complex structures. These findings provide a benchmark comparison of clustering methods for protein sequence data, offering practical insights into algorithm selection based on data characteristics and performance trade-offs.

Spectral clustering (SC) methods have been successfully applied to many real-world applications. The success of these SC methods is largely based on the manifold assumption, namely, that two nearby data points in the high-density region of a low-dimensional data manifold have the same cluster label. However, such an assumption might not always hold on high-dimensional data. When the data do not exhibit a clear low-dimensional manifold structure (e.g., high-dimensional and sparse data), the clustering performance of SC will be degraded and become even worse than K -means clustering. In this paper, motivated by the observation that the true cluster assignment matrix for high-dimensional data can be always embedded in a linear space spanned by the data, we propose the spectral embedded clustering (SEC) framework, in which a linearity regularization is explicitly added into the objective function of SC methods. More importantly, the proposed SEC framework can naturally deal with out-of-sample data. We also present a new Laplacian matrix constructed from a local regression of each pattern and incorporate it into our SEC framework to capture both local and global discriminative information for clustering. Comprehensive experiments on eight real-world high-dimensional datasets demonstrate the effectiveness and advantages of our SEC framework over existing SC methods and K-means-based clustering methods. Our SEC framework significantly outperforms SC using the Nyström algorithm on unseen data.

In this paper, we focus on face clustering in videos. To promote the performance of video clustering by multiple intrinsic cues, i.e., pairwise constraints and multiple views, we propose a constrained multi-view video face clustering method under a unified graph-based model. First, unlike most existing video face clustering methods which only employ these constraints in the clustering step, we strengthen the pairwise constraints through the whole video face clustering framework, both in sparse subspace representation and spectral clustering. In the constrained sparse subspace representation, the sparse representation is forced to explore unknown relationships. In the constrained spectral clustering, the constraints are used to guide for learning more reasonable new representations. Second, our method considers both the video face pairwise constraints as well as the multi-view consistence simultaneously. In particular, the graph regularization enforces the pairwise constraints to be respected and the co-regularization penalizes the disagreement among different graphs of multiple views. Experiments on three real-world video benchmark data sets demonstrate the significant improvements of our method over the state-of-the-art methods.

Spectral clustering has become one of the most popular clustering methods due to its superior performance compared to the traditional clustering methods. However, the performance of spectral clustering would be limited by complex data, such as a huge number of samples and high dimensionality. To address this problem, some existing methods apply deep learning to learn the lower-dimensional representations, spectral clustering is then applied to the representations. Different from the existing methods that separate the two stages of feature representation learning and spectral clustering, in this paper, we propose Spectral Clustering Joint Deep Embedding (SCJDE), a method that simultaneously learns the feature representations and the spectral embedding of spectral clustering via a deep autoencoder. Moreover, a sparsity constraint is imposed to generate better spectral embedding for spectral clustering. Finally, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$k$</tex> -means is performed on the spectral embedding to obtain clustering result. The proposed method can learn a good spectral embedding for spectral clustering by deep learning to obtain better clustering results. The experimental results on both synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

Spectral clustering requires the time-consuming decomposition of the Laplacian matrix of the similarity graph, thus limiting its applicability to large datasets. To improve the efficiency of spectral clustering, a top-down approach was recently proposed, which first divides the data into several micro-clusters (granular-balls), then splits these micro-clusters when they are not ``compact'', and finally uses these micro-clusters as nodes to construct a similarity graph for more efficient spectral clustering. However, this top-down approach is challenging to adapt to unevenly distributed or structurally complex data. This is because constructing micro-clusters as a rough ball struggles to capture the shape and structure of data in a local range, and the simplistic splitting rule that solely targets ``compactness'' is susceptible to noise and variations in data density and leads to micro-clusters with varying shapes, making it challenging to accurately measure the similarity between them. To resolve these issues and improve spectral clustering, this paper first proposes to start from local structures to obtain micro-clusters, such that the complex structural information inside local neighborhoods is well captured by them. Moreover, by noting that Euclidean distance is more suitable for convex sets, this paper further proposes a data splitting rule that couples local density and data manifold structures, so that the similarities of the obtained micro-clusters can be easily characterized. A novel similarity measure between micro-clusters is then proposed for the final spectral clustering. A series of experiments based on synthetic and real-world datasets demonstrate that the proposed method has better adaptability to structurally complex data than granular-ball based methods.

Latent tree models for rounding in spectral clustering

A robust adaptive clustering analysis method for automatic identification of clusters

We consider the problem of image representation and clustering. Traditionally, an n1 x n2 image is represented by a vector in the Euclidean space R n1 x n2. Some learning algorithms are then applied to these vectors in such a high dimensional space for dimensionality reduction, classification, and clustering. However, an image is intrinsically a matrix, or the second order tensor. The vector representation of the images ignores the spatial relationships between the pixels in an image. In this paper, we introduce a tensor framework for image analysis. We represent the images as points in the tensor space Rn1 mathcal Rn2 which is a tensor product of two vector spaces. Based on the tensor representation, we propose a novel image representation and clustering algorithm which explicitly considers the manifold structure of the tensor space. By preserving the local structure of the data manifold, we can obtain a tensor subspace which is optimal for data representation in the sense of local isometry. We call it TensorImage approach. Traditional clustering algorithm such as k-means is then applied in the tensor subspace. Our algorithm shares many of the data representation and clustering properties of other techniques such as Locality Preserving Projections, Laplacian Eigenmaps, and spectral clustering, yet our algorithm is much more computationally efficient. Experimental results show the efficiency and effectiveness of our algorithm.

Machine learning, particularly classification algorithms, has been widely employed for diagnosing COVID-19 cases. However, these methods typically rely on labeled datasets and analyze a single data view. With the vast amount of patient data available without labels, this paper addresses the novel challenge of unsupervised COVID-19 diagnosis. The goal is to harness the abundant data without labels effectively. In recent times, multi-view clustering has garnered considerable attention in the research community. Spectral clustering, known for its robust theoretical framework, is a key focus. However, traditional spectral clustering methods generate only nonlinear data projections, necessitating additional clustering steps. The quality of these post-processing steps can be influenced by various factors, such as initialization procedures and outliers. This paper introduces an enhanced version of the recent “Multiview Spectral Clustering via integrating Nonnegative Embedding and Spectral Embedding” method. While retaining the benefits of the original technique, the proposed model integrates two essential constraints: (1) a constraint for ensuring the consistent smoothness of the nonnegative embedding across all views and (2) an orthogonality constraint imposed on the columns of the nonnegative embedding matrix. The effectiveness of this approach is demonstrated using COVIDx datasets. Additionally, the method is evaluated on other image datasets to validate its suitability for this study.

Improving clustering with constrained communities