Spectral Clustering of Mixed-Type Data

Spectral clustering has become one of the most popular clustering methods due to its superior performance compared to the traditional clustering methods. However, the performance of spectral clustering would be limited by complex data, such as a huge number of samples and high dimensionality. To address this problem, some existing methods apply deep learning to learn the lower-dimensional representations, spectral clustering is then applied to the representations. Different from the existing methods that separate the two stages of feature representation learning and spectral clustering, in this paper, we propose Spectral Clustering Joint Deep Embedding (SCJDE), a method that simultaneously learns the feature representations and the spectral embedding of spectral clustering via a deep autoencoder. Moreover, a sparsity constraint is imposed to generate better spectral embedding for spectral clustering. Finally, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$k$</tex> -means is performed on the spectral embedding to obtain clustering result. The proposed method can learn a good spectral embedding for spectral clustering by deep learning to obtain better clustering results. The experimental results on both synthetic and real-world datasets demonstrate the effectiveness of the proposed method.

Spectral clustering (SC) is one of the most popular clustering methods and often outperforms traditional clustering methods. SC uses the eigenvectors of a Laplacian matrix calculated from a similarity matrix of a dataset. SC has serious drawbacks: the significant increases in the time complexity derived from the computation of eigenvectors and the memory space complexity to store the similarity matrix. To address the issues, I develop a new approximate spectral clustering using the network generated by growing neural gas (GNG), called ASC with GNG in this study. ASC with GNG uses not only reference vectors for vector quantization but also the topology of the network for extraction of the topological relationship between data points in a dataset. ASC with GNG calculates the similarity matrix from both the reference vectors and the topology of the network generated by GNG. Using the network generated from a dataset by GNG, ASC with GNG achieves to reduce the computational and space complexities and improve clustering quality. In this study, I demonstrate that ASC with GNG effectively reduces the computational time. Moreover, this study shows that ASC with GNG provides equal to or better clustering performance than SC.

ABSTRACTConventional statistical process control tools target either continuous or categorical data but seldom both at the same time. However, mixed-type data consisting of both continuous and categorical observations are becoming more common in modern manufacturing processes and service management. However, they cannot be analyzed using traditional methods. By assuming that there is a latent continuous variable that determines the attribute levels of a categorical variable, the ordinal information among the attribute levels can be exploited. This enables us to simultaneously describe and monitor continuous and categorical data in a unified framework of standardized ranks, based on which a multivariate exponentially weighted moving average control chart is proposed. This control chart specializes in detecting location shifts in continuous data and in latent continuous distributions of categorical data. Numerical simulations show that our proposed chart can efficiently detect location shifts and is robust to various distributions.

Recently, a large amount of work has been devoted to the study of spectral clustering—a simple yet powerful method for finding structure in a data set using spectral properties of an associated pairwise similarity matrix. Most of the existing spectral clustering algorithms estimate only one cluster number or estimate non-unique cluster numbers based on eigengap criterion. However, the number of clusters not always exists one, and eigengap criterion lacks theoretical justification. In this paper, we propose non-unique cluster numbers determination methods based on stability in spectral clustering (NCNDBS). We first utilize the multiway normalized cut spectral clustering algorithm to cluster data set for a candidate cluster number \(k\). Then the ratio value of the multiway normalized cut criterion of the obtained clusters and the sum of the leading eigenvalues (descending sort) of the stochastic transition matrix is chosen as a standard to decide whether the \(k\) is a reasonable cluster number. At last, by varying the scaling parameter in the Gaussian function, we judge whether the reasonable cluster number \(k\) is also a stability one. By three stages, we can determine non-unique cluster numbers of a data set. The Lumpability theorem concluded by Meil\(\breve{a}\) and Xu provides a theoretical base for our methods. NCNDBS can estimate non-unique cluster numbers of the data set successfully by illustrative experiments.

Data mining and knowledge discovery aim at finding useful information from typically massive collections of data, and then extracting useful knowledge from the information. To date a large number of approaches have been proposed to find useful information and discover useful knowledge; for example, decision trees, Bayesian belief networks, evidence theory, rough set theory, fuzzy set theory, kNN (k-nearest-neighborhood) classifier, neural networks, and support vector machines. However, these approaches are based on a specific data type. In the real world, an intelligent system often encounters mixed data types, incomplete information (missing values), and imprecise information (fuzzy conditions). In the UCI (University of California – Irvine) Machine Learning Repository, it can be seen that there are many real world data sets with missing values and mixed data types. It is a challenge to enable machine learning or data mining approaches to deal with mixed data types (Ching, 1995; Coppock, 2003) because there are difficulties in finding a measure of similarity between objects with mixed data type attributes. The problem with mixed data types is a long-standing issue faced in data mining. The emerging techniques targeted at this issue can be classified into three classes as follows: (1) Symbolic data mining approaches plus different discretizers (e.g., Dougherty et al., 1995; Wu, 1996; Kurgan et al., 2004; Diday, 2004; Darmont et al., 2006; Wu et al., 2007) for transformation from continuous data to symbolic data; (2) Numerical data mining approaches plus transformation from symbolic data to numerical data (e.g.,, Kasabov, 2003; Darmont et al., 2006; Hadzic et al., 2007); (3) Hybrid of symbolic data mining approaches and numerical data mining approaches (e.g.,, Tung, 2002; Kasabov, 2003; Leng et al., 2005; Wu et al., 2006). Since hybrid approaches have the potential to exploit the advantages from both symbolic data mining and numerical data mining approaches, this chapter, after discassing the merits and shortcomings of current approaches, focuses on applying Self-Organizing Computing Network Model to construct a hybrid system to solve the problems of knowledge discovery from databases with a diversity of data types. Future trends for data mining on mixed type data are then discussed. Finally a conclusion is presented.

Spectral clustering (SC) is a large family of grouping methods that partition data using eigenvectors of an affinity matrix derived from the data. Though SC methods have been successfully applied to a large number of challenging clustering scenarios, it is noteworthy that they will fail when there are significant intersections among different clusters. In this paper, based on the analysis that SC methods are able to work well when the affinity values of the points belonging to different clusters are relatively low, we propose a new method, called spectral multi-manifold clustering (SMMC), which is able to handle intersections. In our model, the data are assumed to lie on or close to multiple smooth low-dimensional manifolds, where some data manifolds are separated but some are intersecting. Then, local geometric information of the sampled data is incorporated to construct a suitable affinity matrix. Finally, spectral method is applied to this affinity matrix to group the data. Extensive experiments on synthetic as well as real datasets demonstrate the promising performance of SMMC.

Fuzzy spectral clustering with robust spatial information for image segmentation

Fast spectral clustering method based on graph similarity matrix completion

Mixed-type data consisting of both continuous observations and categorical observations are becoming prevalent in manufacturing processes and service management. The majority of existing statistical process control tools are designed to monitor either continuous data or categorical data but seldom both. In this article, we propose a directional exponentially weighted moving average control scheme composed of monitoring and diagnosis for mixed-type data. We assume that there is a latent unknown continuous distribution that determines the attribute levels of a categorical variable, and represent both continuous data and categorical data by standardised ranks. The proposed control chart also incorporates directional information to facilitate diagnosing the shift direction. Monte Carlo simulations demonstrate the efficiency of the proposed control scheme.

Spectral clustering (SC) has been successfully used in the field of machine learning and data mining. Although it has been widely used, one significant drawback of SC is its expensive computation cost. As data size and dimensions increase, the time overhead of SC is unbearable. In order to overcome this issue, we propose a novel SC algorithm KSC to reduce the time cost. In this paper, we use Markov chain Monte Carlo sampling to approximate the seeding step of k-means++, and get the initial cluster centers of k-means. At the same time, we redefine the data set of the SC by calculating the distance from the sample point to the center point to achieve dimensionality reduction to reduce time overhead. At last, we perform extensive experiments comparing KSC against other methods both real and synthetic datasets. Our results show that KSC has significantly improved its speed while maintaining clustering accuracy.

A powerful method in knowledge discovery and cluster extraction is the use of self-organizing maps (SOMs), which provide adaptive quantization of the data together with its topologically ordered lower-dimensional representation on a rigid lattice. The knowledge extraction from SOMs is often performed interactively from informative visualizations. Even though interactive cluster extraction is successful, it is often time consuming and usually not straightforward for inexperienced users. In order to cope with the need of fast and accurate analysis of increasing amount of data, automated methods for SOM clustering have been popular. In this study, we use spectral clustering, a graph partitioning method based on eigenvector decomposition, for automated clustering of the SOM. Experimental results based on seven real data sets indicate that spectral clustering can successfully be used as an automated SOM segmentation tool, and it outperforms hierarchical clustering methods with distance based similarity measures.KeywordsSpectral ClusterHierarchical Agglomerative ClusterEigenvector DecompositionHierarchical Agglomerative Cluster MethodCluster ExtractionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Simulated annealing spectral clustering algorithm for image segmentation

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

Spectral clustering has been successfully used in the field of pattern recognition and image processing. The efficiency of spectral clustering, however, depends heavily on the similarity measure adopted. A widely used similarity measure is the Gaussian kernel function where Euclidean distance is used. Unfortunately, the Gaussian kernel function is parameter sensitive and the Euclidean distance is usually not suitable to the complex distribution data. In this paper, a novel similarity measure called kernel fuzzy similarity measure is proposed first, Then this novel measure is integrated into spectral clustering to get a new clustering method: kernel fuzzy similarity based spectral clustering (KFSC). To alleviate the computational complexity of KFSC on image segmentation, Nystr\(\ddot{o}\)m method is used in KFSC. At last, the experiments on three synthetic texture images are made, and the results demonstrate the effectiveness of the proposed algorithm.

Brain tumour segmentation from MRI using superpixels based spectral clustering