Fair Spectral Clustering Based on Coordinate Descent

Spectral clustering (SC) has been widely used in many applications and shows excellent performance. Its high computational cost limits its applications; many strategies including the anchor technique can partly alleviate the high computational cost problem. However, early methods ignore the fact that SC usually involves two stages: relaxation and postprocessing, i.e., it relaxes the discrete constraints to continuous constraints, and then conducts the postprocessing to get the discrete solution, which is time-consuming and deviates from directly solving the primal problem. In this article, we first adopt the bipartite graph strategy to reduce the time complexity of SC, and then an improved coordinate descent (CD) method is proposed to solve the primal problem directly without singular value decomposition (SVD) and postprocessing, i.e., directly solving the primal problem not approximately solving. Experiments on various real-world benchmark datasets show that the proposed method can get better solutions faster with better clustering performance than traditional optimization methods. Furthermore, it can jump out of local minima of traditional methods and continue to obtain better local solutions. Moreover, compared with other clustering methods, it also shows its superiority.

Spectral Clustering (SC) is an effective clustering method for its excellent performance in partitioning non-linearly distributed data. On the other hand, Ensemble Clustering (EC), a different clustering technology, can promote cluster quality by ensembling the results of base clusterings. In this work, we concentrate on an EC framework that utilizes SC as the base method. Nevertheless, SC suffers from scalability due to its high computational complexity in constructing the Laplacian graph and computing the corresponding eigendecomposition. In the past decades, many efforts have been made to it. However, SC suffers from the scalability issue in processing extensive data, especially in web-scale scenarios. Additionally, EC requires multiple clustering results as the ensemble bases, which further aggravates resource consumption. To address this issue, LiteWSEC, a simple yet efficient Lightweight Framework for Web-scale Spectral Ensemble Clustering, is proposed to cluster web-scale data with limited resource requirements. It adopts the Web-scale Spectral Clustering (WSC) as the base method, which has minimal space overhead without computing overall embedding explicitly. LiteWSEC is highly flexible in the memory requirement, which is adaptive to the available resource. It can partition web-scale data (e.g., <inline-formula><tex-math notation="LaTeX">$n $</tex-math></inline-formula> = 8,000 k) in an resource-limited host (e.g., memory is restricted to 1 GB). Experiments on real-world, large-scale, and web-scale datasets demonstrate both the efficiency and effectiveness of LiteWSEC over state-of-the-art SC and EC methods.

Spectral clustering has become one of the most popular clustering methods due to its superior performance compared to the traditional clustering methods. However, the performance of spectral clustering would be limited by complex data, such as a huge number of samples and high dimensionality. To address this problem, some existing methods apply deep learning to learn the lower-dimensional representations, spectral clustering is then applied to the representations. Different from the existing methods that separate the two stages of feature representation learning and spectral clustering, in this paper, we propose Spectral Clustering Joint Deep Embedding (SCJDE), a method that simultaneously learns the feature representations and the spectral embedding of spectral clustering via a deep autoencoder. Moreover, a sparsity constraint is imposed to generate better spectral embedding for spectral clustering. Finally, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$k$</tex> -means is performed on the spectral embedding to obtain clustering result. The proposed method can learn a good spectral embedding for spectral clustering by deep learning to obtain better clustering results. The experimental results on both synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

Spectral clustering algorithms rely on graphs where edges are defined based on the similarity between the vertices (data points). The effectiveness and fairness of spectral clustering depend significantly on how the graph is constructed. While automated graph construction methods, which learn graphs from real-valued vector datasets, have demonstrated strong performance in the quality of clustering, fairness concerns still remain. In this work, we introduce a graph construction method that incorporates a new fairness definition—Edge Disparate Impact—into the edge relationships, aiming to produce a fair graph. This approach modifies the optimization process of automated graph construction to account for fairness, resulting in a more equitable graph. Extensive experiments were conducted to compare our method with the latest graph construction techniques and fair spectral clustering algorithms. The results prove that, by using a fair graph for spectral clustering, fairness is improved in the resulting clusters. We also demonstrate that our method outperforms baseline approaches in both fairness and the quality of clustering.

A robust adaptive clustering analysis method for automatic identification of clusters

Spectral clustering has become a typical and efficient clustering method used in a variety of applications. The critical step of spectral clustering is the similarity measurement, which largely determines the performance of the spectral clustering method. In this paper, we propose a novel spectral clustering algorithm based on the local similarity measure of shared neighbors. This similarity measurement exploits the local density information between data points based on the weight of the shared neighbors in a directed k $$ k $$ -nearest neighbor graph with only one parameter k $$ k $$ , that is, the number of nearest neighbors. Numerical experiments on synthetic and real-world datasets demonstrate that our proposed algorithm outperforms other existing spectral clustering algorithms in terms of the clustering performance measured via the normalized mutual information, clustering accuracy, and F $$ F $$ -measure. As an example, the proposed method can provide an improvement of 15.82% in the clustering performance for the Soybean dataset.

Spectral clustering (SC) is one of the most popular clustering methods and often outperforms traditional clustering methods. SC uses the eigenvectors of a Laplacian matrix calculated from a similarity matrix of a dataset. SC has serious drawbacks: the significant increases in the time complexity derived from the computation of eigenvectors and the memory space complexity to store the similarity matrix. To address the issues, I develop a new approximate spectral clustering using the network generated by growing neural gas (GNG), called ASC with GNG in this study. ASC with GNG uses not only reference vectors for vector quantization but also the topology of the network for extraction of the topological relationship between data points in a dataset. ASC with GNG calculates the similarity matrix from both the reference vectors and the topology of the network generated by GNG. Using the network generated from a dataset by GNG, ASC with GNG achieves to reduce the computational and space complexities and improve clustering quality. In this study, I demonstrate that ASC with GNG effectively reduces the computational time. Moreover, this study shows that ASC with GNG provides equal to or better clustering performance than SC.

The spectral clustering algorithm's space complexity is O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), while time complexity is O(n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ). When dealing with large amounts of data, the memory will overflow and run-time is too long. For the general problem of spectral clustering, if the clustering data of sub-problem between the original problem has the same probability distribution, it can be applied to divide and conquer strategy for the problem of spectral clustering, by the spectral clustering results of sub-problems to get the spectral clustering results of original problem. To spectral clustering image segmentation as a research object, we will discuss the divide and conquer strategy for spectral clustering in this paper. Experiments show that the application of divide and conquer method for spectral clustering image segmentation, we can get a perfect performance in image segmentation.

Spectral ensemble clustering with doubly stochastic co-association matrix

Spectral clustering is a very popular graph-based clustering technique that partitions data groups based on the input data similarity matrix. Many past studies based on spectral clustering, however, do not consider the global discriminative structure of the dataset. Also, the benefits of using more than one kernel have not been fully exploited with respect to spectral clustering, although it has been established by these past studies that using more than one kernel in clustering can result in a more accurate clustering than those obtained with a single kernel. Multi-kernel approaches, however, tend to be more time consuming compared to single kernel methods. To compensate these drawbacks, we integrate a global discriminative term into the clustering with an adaptive neighbor framework. This is done to preserve both the global geometric information and global discriminative information in a dual kernel space, in an attempt to optimize clustering performance. Via co-regularization, we utilize more than one kernel space to take advantage of the benefits of multiple kernels. We, however, use two heterogeneous kernels to help us reduce clustering time, since the ability to quickly process data is as equally important as its accuracy in this era of information explosion. Since these different kernel spaces admit the same underlying clustering of the data, we approach the problem looking for clustering consistent across the two kernel views. Hence we are able to detect the non-linear intrinsic geometrical information of the dataset. We perform clustering using the obtained indicator matrix from our modified Laplacian utilizing k-means. Our Experimental outcomes show that our approach gives satisfactory results in terms of accuracy and NMI, with time-to-cluster savings in comparison to other state-of-the-art clustering methods using both synthetic and public datasets.

Protein sequence data analysis is a fundamental task in bioinformatics, supporting the exploration of biological variations and the identification of functional relationships among proteins. This study presents a performance analysis of four clustering algorithms, which include Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Agglomerative Hierarchical Clustering, and Spectral Clustering, applied to protein sequence datasets. Feature extraction was conducted using the Discere package in Python, generating 27 numerical attributes from protein sequences. The optimal number of clusters for BIRCH, Agglomerative, and Spectral Clustering was determined using the Elbow method, while DBSCAN parameters (MinPts, Eps) were tuned using the sorted k-distance plot. Clustering performance was assessed using the Silhouette Score. Among the algorithms, DBSCAN produced the highest silhouette score of 0.8105, whereas BIRCH achieved a strong balance between clustering quality, with a score of 0.7405, and computational efficiency. Agglomerative clustering provided moderate results with a score of 0.6779, while Spectral clustering yielded the lowest score of 0.6310 but demonstrated flexibility in capturing complex structures. These findings provide a benchmark comparison of clustering methods for protein sequence data, offering practical insights into algorithm selection based on data characteristics and performance trade-offs.

Coordinate Descent Optimized Trace Difference Model for Joint Clustering and Feature Extraction

Spectral clustering (SC) has been successfully used in the field of machine learning and data mining. Although it has been widely used, one significant drawback of SC is its expensive computation cost. As data size and dimensions increase, the time overhead of SC is unbearable. In order to overcome this issue, we propose a novel SC algorithm KSC to reduce the time cost. In this paper, we use Markov chain Monte Carlo sampling to approximate the seeding step of k-means++, and get the initial cluster centers of k-means. At the same time, we redefine the data set of the SC by calculating the distance from the sample point to the center point to achieve dimensionality reduction to reduce time overhead. At last, we perform extensive experiments comparing KSC against other methods both real and synthetic datasets. Our results show that KSC has significantly improved its speed while maintaining clustering accuracy.

Spectral clustering has been shown to be more effective than most of the traditional clustering algorithms. However, the heavy computational cost of spectral clustering limits its applicability to large-scale clustering problems. To perform spectral clustering on large datasets, in this paper, we propose an accelerated spectral clustering method based on sparse presentation where each data point is presented as sparse linear combinations of a part of representative data points. The hubs that appear frequently in the nearest neighbor lists of other data points are selected as the representative data points, by which a proper spectral embedding is constructed. Taking advantage of the topological property of hubs, the proposed method is able to achieve scalable and accurate clustering results. We evaluated the proposed method on both synthetic and real-world datasets to show its effectiveness in comparison to the existing related methods.

Network Intrusion Detection Systems (NIDSs) are an increasingly important tool for the prevention and mitigation of cyber attacks. Over the past years, a lot of research efforts have aimed at leveraging the increasingly powerful models of Machine Learning (ML) for this purpose. A number of labelled synthetic datasets have been generated and made publicly available by researchers, and they have become the benchmarks via which new ML-based NIDS classifiers are being evaluated. Recently published results show excellent classification performance with these datasets, increasingly approaching 100 percent performance across key evaluation metrics such as Accuracy, F1 score, AUC, etc. Unfortunately, we have not yet seen these excellent academic research results translated into practical NIDS systems with such near-perfect performance. This motivated our research presented in this paper, where we analyse the statistical properties of the benign traffic in three of the more recent and relevant NIDS datasets, (CIC_IDS, UNSW_NB15, TON_IOT), by converting them into a common flow format. As a comparison, we consider two datasets obtained from real-world production networks, one from a university network and one from a medium size Internet Service Provider (ISP). Our results show that the two real-world datasets are quite similar among themselves in regards to most of the considered statistical features. Equally, the three synthetic datasets are also relatively similar within their group. However, and most importantly, our results show a distinct difference of most of the considered statistical features between the three synthetic datasets and the two real-world datasets. Since ML relies on the basic assumption of training and test datasets being sampled from the same distribution, this raises the question of how well the performance results of ML-classifiers trained on the considered synthetic datasets can translate and generalise to real-world networks. We believe this is an interesting and relevant question which provides motivation for further research in this space.