Sparse and Low-Rank Subspace Data Clustering with Manifold Regularization Learned by Local Linear Embedding

We propose Ordered Subspace Clustering (OSC) to segment data drawn from a sequentially ordered union of subspaces. Current subspace clustering techniques learn the relationships within a set of data and then use a separate clustering algorithm such as NCut for final segmentation. In contrast our technique, under certain conditions, is capable of segmenting clusters intrinsically without providing the number of clusters as a parameter. Similar to Sparse Subspace Clustering (SSC) we formulate the problem as one of finding a sparse representation but include a new penalty term to take care of sequential data. We test our method on data drawn from infrared hyper spectral data, video sequences and face images. Our experiments show that our method, OSC, outperforms the state of the art methods: Spatial Subspace Clustering (SpatSC), Low-Rank Representation (LRR) and SSC.

Subspace clustering methods with sparsity prior, such as Sparse Subspace Clustering (SSC) [1], are effective in partitioning the data that lie in a union of subspaces. Most of those methods require certain assumptions, e.g. independence or disjointness, on the subspaces. These assumptions are not guaranteed to hold in practice and they limit the application of existing sparse subspace clustering methods. In this paper, we propose \(\ell ^{0}\)-induced sparse subspace clustering (\(\ell ^{0}\)-SSC). In contrast to the required assumptions, such as independence or disjointness, on subspaces for most existing sparse subspace clustering methods, we prove that subspace-sparse representation, a key element in subspace clustering, can be obtained by \(\ell ^{0}\)-SSC for arbitrary distinct underlying subspaces almost surely under the mild i.i.d. assumption on the data generation. We also present the “no free lunch” theorem that obtaining the subspace representation under our general assumptions can not be much computationally cheaper than solving the corresponding \(\ell ^{0}\) problem of \(\ell ^{0}\)-SSC. We develop a novel approximate algorithm named Approximate \(\ell ^{0}\)-SSC (\(\hbox {A}\ell ^{0}\)-SSC) that employs proximal gradient descent to obtain a sub-optimal solution to the optimization problem of \(\ell ^{0}\)-SSC with theoretical guarantee, and the sub-optimal solution is used to build a sparse similarity matrix for clustering. Extensive experimental results on various data sets demonstrate the superiority of \(\hbox {A}\ell ^{0}\)-SSC compared to other competing clustering methods.

This paper studies the subspace segmentation problem. Given a set of data points drawn from a union of subspaces, the goal is to partition them into their underlying subspaces they were drawn from. The spectral clustering method is used as the framework. It requires to find an affinity matrix which is close to block diagonal, with nonzero entries corresponding to the data point pairs from the same subspace. In this work, we argue that both sparsity and the grouping effect are important for subspace segmentation. A sparse affinity matrix tends to be block diagonal, with less connections between data points from different subspaces. The grouping effect ensures that the highly corrected data which are usually from the same subspace can be grouped together. Sparse Subspace Clustering (SSC), by using $\ell^1$-minimization, encourages sparsity for data selection, but it lacks of the grouping effect. On the contrary, Low-Rank Representation (LRR), by rank minimization, and Least Squares Regression (LSR), by $\ell^2$-regularization, exhibit strong grouping effect, but they are short in subset selection. Thus the obtained affinity matrix is usually very sparse by SSC, yet very dense by LRR and LSR. In this work, we propose the Correlation Adaptive Subspace Segmentation (CASS) method by using trace Lasso. CASS is a data correlation dependent method which simultaneously performs automatic data selection and groups correlated data together. It can be regarded as a method which adaptively balances SSC and LSR. Both theoretical and experimental results show the effectiveness of CASS.

Attention reweighted sparse subspace clustering

Gene expression data is a kind of high dimension and small sample size data. The clustering accuracy of conventional clustering techniques is lower on gene expression data due to its high dimension. Because some subspace segmentation approaches can be better applied in the high-dimensional space, three new subspace clustering models for gene expression data sets are proposed in this work. The proposed projection subspace clustering models have projection sparse subspace clustering, projection low-rank representation subspace clustering and projection least-squares regression subspace clustering which combine projection technique with sparse subspace clustering, low-rank representation and least-square regression, respectively. In order to compute the inner product in the high-dimensional space, the kernel function is used to the projection subspace clustering models. The experimental results on six gene expression data sets show these models are effective.

In the analysis of single-cell RNA-sequencing (scRNA-seq) data, how to effectively and accurately identify cell clusters from a large number of cell mixtures is still a challenge. Low-rank representation (LRR) method has achieved excellent results in subspace clustering. But in previous studies, most LRR-based methods usually choose the original data matrix as the dictionary. In addition, the methods based on LRR usually use spectral clustering algorithm to complete cell clustering. Therefore, there is a matching problem between the spectral clustering method and the affinity matrix, which is difficult to ensure the optimal effect of clustering. Considering the above two points, we propose the DLNLRR method to better identify the cell type. First, DLNLRR can update the dictionary during the optimization process instead of using the predefined fixed dictionary, so it can realize dictionary learning and LRR learning at the same time. Second, DLNLRR can realize subspace clustering without relying on spectral clustering algorithm, that is, we can perform clustering directly based on the low-rank matrix. Finally, we carry out a large number of experiments on real single-cell datasets and experimental results show that DLNLRR is superior to other scRNA-seq data analysis algorithms in cell type identification.

The data mining experiments from this chapter are a component of a larger research which is to be used to make several correlations, analysis in order to integrate data mining process in the managerial system for optimal decision support. I offered an insight of how data mining processes are being applied in the large spectrum of education by presenting recent applications and studies published in the scientific literature, considered to be relevant to the development of this emerging science. I presented my work through a number of experiments conducted over questionnaires data and scholastic databases at Faculty of Economics and Business Administration Cluj-Napoca, using classification learning and data clustering methods. In the last part of the chapter I introduced the concept of Academic Intelligence Management, and illustrated the integration of data mining processes and their particular role in higher education management and decision support. The studies will continue with deeper mining of educational topics, such as performance in scientific research, correlations between the students' knowledge and the competences demanded on the labor market, academic failure, to perceive what and how much the students know, to realize learning gaps, and also improve teaching methods and educational management processes.

Low Rank Representation (LRR) based unsupervised clustering methods have achieved great success since these methods could explore low-dimensional subspace structure embedded in original data effectively. The conventional LRR methods generally treat the data as the points in Euclidean space. However, it is no longer suitable for high-dimension data (such as video or imageset). That is because high-dimension data are always considered as non-linear manifold data such as Grassmann manifold. Besides, the typical LRR methods always adopt the traditional single nuclear norm based low rank constraint which can not fully reveal the low rank property of the data representation and often leads to suboptimal solution. In this paper, a new LRR based clustering model is constructed on Grassmann manifold for high-dimension data. In the proposed method, each high-dimension data is formed as a sample on Grassmann manifold with non-linear metric. Meanwhile, a non-convex low rank representation is adopt to reveal the intrinsic property of these high-dimension data and reweighted rank minimization constraint is introduced. The experimental results on several public datasets show that the proposed method outperforms the state-of-the-art clustering methods.

Multiview subspace clustering is one of the most widely used methods for exploiting the internal structures of multiview data. Most previous studies have performed the task of learning multiview representations by individually constructing an affinity matrix for each view without simultaneously exploiting the intrinsic characteristics of multiview data. In this article, we propose a multiview low-rank representation (MLRR) method to comprehensively discover the correlation of multiview data for multiview subspace clustering. MLRR considers symmetric low-rank representations (LRRs) to be an approximately linear spatial transformation under the new base, that is, the multiview data themselves, to fully exploit the angular information of the principal directions of LRRs, which is adopted to construct an affinity matrix for multiview subspace clustering, under a symmetric condition. MLRR takes full advantage of LRR techniques and a diversity regularization term to exploit the diversity and consistency of multiple views, respectively, and this method simultaneously imposes a symmetry constraint on LRRs. Hence, the angular information of the principal directions of rows is consistent with that of columns in symmetric LRRs. The MLRR model can be efficiently calculated by solving a convex optimization problem. Moreover, we present an intuitive fusion strategy for symmetric LRRs from the perspective of spectral clustering to obtain a compact representation, which can be shared by multiple views and comprehensively represents the intrinsic features of multiview data. Finally, the experimental results based on benchmark datasets demonstrate the effectiveness and robustness of MLRR compared with several state-of-the-art multiview subspace clustering algorithms.

Sparse subspace clustering (SSC) has achieved state‐of‐the‐art clustering quality by performing spectral clustering over an ℓ1‐norm based similarity graph. However, SSC is a transductive method, i.e. it cannot handle out‐of‐sample data that is not used to construct the graph. For each new datum, SSC requires solving n optimisation problems in O(n) variables, where n is the number of data points. Therefore, it is inefficient to apply SSC in fast online clustering and scalable grouping. An inductive spectral clustering algorithm called inductive SSC (iSSC) is proposed, which makes SSC feasible to cluster out‐of‐sample data. iSSC adopts the assumption that high‐dimensional data actually lie on the low‐dimensional manifold such that out‐of‐sample data could be grouped in the embedding space learned from in‐sample data. Experimental results show that iSSC is promising in clustering out‐of‐sample data.

Subspace clustering is an effective algorithm refer to the problem which separate the data lying on a union of subspaces. Usually the algorithm consists of two steps. First, an affinity matrix is calculated from the self-representation of the data. Second, spectral clustering is used to cluster the data by the affinity matrix. The paper introduces a new method based on the sparse subspace clustering. The new method used a new objective which considered both the sparseness and grouping effect. And the affinity matrix which is calculated from the self-representation of the data is furthered optimized. It proved to be more efficient than existing subspace clustering methods.

Subspace clustering based on latent low rank representation with Frobenius norm minimization

Understanding the relationship between brain function and structure is vital important in the field of brain image analysis. It elucidates the working mechanism of the brain, which will contribute to better understand the brain and simulate the brain-like system. Extensive efforts have been made on this topic, but still far from the satisfactory. The major difficulties are at least two aspects. One is the huge individual difference among the subjects, which makes it hard to obtain stable results at groupwise level, e.g., noise signals can significantly affect the exploring process. The other one is the huge difference between functional and structural features of the brain, both in their pattern and size, which are very different. To alleviate the above problems, in this paper, we propose a two-stage multi-view low-rank sparse subspace clustering (Two-stage MLRSSC) method to jointly study the relationship between brain function and structure and identify the common regions of brain function and structure. The major innovation of proposed Two-stage MLRSSC is that comparable features of brain function and structure can be effectively extracted from low-rank sparse representation, and results are further improved the stability by two-stage strategy. Finally, groupwise-based stable functional and structural common regions are identified for better understanding the relationship. Experimental results shed new ways to explore the brain function and structure, new insights are observed and discussed.KeywordsBrain functionBrain structureMulti-ViewLow-Rank Sparse Subspace Clustering

Biomedical cell image contains different types of components. The segmentation of these components is essential for characterizing the overall structure of leucocytes. However, the task is challenging because the data generally possess irregular patterns, varying eccentricity and congestion of inter class substructures. In order to perform the automatic segmentation of leucocytes, many approaches in machine learning have been proposed. Most of the recent methods utilize k-means and fuzzy c-means algorithms. These methods suffer from the under-segmentation and the oversegmentation of data points. We have adopted an efficient Low Rank Representation (LRR) method for the segmentation of cell image patterns. Its overall segmentation performance is very high in comparison to the k-means and fuzzy c-means segmentation methods. It discriminates sub structures by incorporating multiple feature data into the joint representation matrix. The main improvements of our adopted approach, include the global geometric representation of different cell patterns and the control of different sources of noise in the data. Data points of the cell image are evolved with the basic assumption as x = m + n. Where x is the input observation, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$m$</tex> is the set of data points that lies in the correct subspace and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex> represent the noise in the data. The performance of the model was evaluated on the different types of medical images. It has reported the clustering accuracy of 97.98% on the biomedical database. Whereas an accuracy of 93.1 % was reported on ALL-IDB cell dataset.

Subspace clustering using a symmetric low-rank representation