$$\ell ^{0}$$-Sparse Subspace Clustering

Attention reweighted sparse subspace clustering

Data clustering is an important research topic in data mining and signal processing communications. In all the data clustering methods, the subspace spectral clustering methods based on self expression model, e.g., the Sparse Subspace Clustering (SSC) and the Low Rank Representation (LRR) methods, have attracted a lot of attention and shown good performance. The key step of SSC and LRR is to construct a proper affinity or similarity matrix of data for spectral clustering. Recently, Laplacian graph constraint was introduced into the basic SSC and LRR and obtained considerable improvement. However, the current graph construction methods do not well exploit and reveal the non-linear properties of the clustering data, which is common for high dimensional data. In this paper, we introduce the classic manifold learning method, the Local Linear Embedding (LLE), to learn the non-linear structure underlying the data and use the learned local geometry of manifold as a regularization for SSC and LRR, which results the proposed LLE-SSC and LLE-LRR clustering methods. Additionally, to solve the complex optimization problem involved in the proposed models, an efficient algorithm is also proposed. We test the proposed data clustering methods on several types of public databases. The experimental results show that our methods outperform typical subspace clustering methods with Laplacian graph constraint.

Subspace clustering is an effective algorithm refer to the problem which separate the data lying on a union of subspaces. Usually the algorithm consists of two steps. First, an affinity matrix is calculated from the self-representation of the data. Second, spectral clustering is used to cluster the data by the affinity matrix. The paper introduces a new method based on the sparse subspace clustering. The new method used a new objective which considered both the sparseness and grouping effect. And the affinity matrix which is calculated from the self-representation of the data is furthered optimized. It proved to be more efficient than existing subspace clustering methods.

Multi-kernel sparse subspace clustering on the Riemannian manifold of symmetric positive definite matrices

This paper studies the subspace segmentation problem. Given a set of data points drawn from a union of subspaces, the goal is to partition them into their underlying subspaces they were drawn from. The spectral clustering method is used as the framework. It requires to find an affinity matrix which is close to block diagonal, with nonzero entries corresponding to the data point pairs from the same subspace. In this work, we argue that both sparsity and the grouping effect are important for subspace segmentation. A sparse affinity matrix tends to be block diagonal, with less connections between data points from different subspaces. The grouping effect ensures that the highly corrected data which are usually from the same subspace can be grouped together. Sparse Subspace Clustering (SSC), by using $\ell^1$-minimization, encourages sparsity for data selection, but it lacks of the grouping effect. On the contrary, Low-Rank Representation (LRR), by rank minimization, and Least Squares Regression (LSR), by $\ell^2$-regularization, exhibit strong grouping effect, but they are short in subset selection. Thus the obtained affinity matrix is usually very sparse by SSC, yet very dense by LRR and LSR. In this work, we propose the Correlation Adaptive Subspace Segmentation (CASS) method by using trace Lasso. CASS is a data correlation dependent method which simultaneously performs automatic data selection and groups correlated data together. It can be regarded as a method which adaptively balances SSC and LSR. Both theoretical and experimental results show the effectiveness of CASS.

We propose Ordered Subspace Clustering (OSC) to segment data drawn from a sequentially ordered union of subspaces. Current subspace clustering techniques learn the relationships within a set of data and then use a separate clustering algorithm such as NCut for final segmentation. In contrast our technique, under certain conditions, is capable of segmenting clusters intrinsically without providing the number of clusters as a parameter. Similar to Sparse Subspace Clustering (SSC) we formulate the problem as one of finding a sparse representation but include a new penalty term to take care of sequential data. We test our method on data drawn from infrared hyper spectral data, video sequences and face images. Our experiments show that our method, OSC, outperforms the state of the art methods: Spatial Subspace Clustering (SpatSC), Low-Rank Representation (LRR) and SSC.

Subspace clustering is the problem of partitioning unlabeled data points into a number of clusters so that data points within one cluster lie approximately on a low-dimensional linear subspace. In many practical scenarios, the dimensionality of data points to be clustered is compressed due to the constraints of measurement, computation, or privacy. In this paper, we study the theoretical properties of a popular subspace clustering algorithm named sparse subspace clustering (SSC) and establish formal success conditions of SSC on dimensionality-reduced data. Our analysis applies to the most general fully deterministic model, where both underlying subspaces and data points within each subspace are deterministically positioned, and also a wide range of dimensionality reduction techniques (e.g., Gaussian random projection, uniform subsampling, and sketching) that fall into a subspace embedding framework. Finally, we apply our analysis to a differentially private SSC algorithm and established both privacy and utility guarantees of the proposed method.

The main contribution of the paper is a new approach to subspace clustering that is significantly more computationally efficient and scalable than existing state-of-the-art methods. The central idea is to modify the regression technique in sparse subspace clustering (SSC) by replacing the $\ell_{1}$ minimization with a generalization called Ordered Weighted $\ell_{1}$ (OWL) minimization which performs simultaneous regression and clustering of correlated variables. Using random geometric graph theory, we prove that OWL regression selects more points within each subspace than $\ell_{1}$, resulting in better clustering results. This allows for accurate subspace clustering based on regression solutions for only a small subset of the total dataset, significantly reducing the computational complexity compared to SSC. In experiments, we find that our OWL approach can achieve a speedup of $ 20\times$ to $30\times$ for synthetic problems and $4\times$ to $8\times$ on real data problems.

Sparse subspace clustering (SSC) has achieved the state-of-the-art performance in the clustering of hyperspectral images (HSIs). However, the high computational complexity and sensitivity to noise limit its clustering performance. In this paper, we propose a scalable SSC method for the large-scale HSIs, which significantly accelerates the clustering speed of SSC without sacrificing clustering accuracy. A small landmark dictionary is first generated by applying k-means to the original data, which results in the significant reduction of the number of optimization variables in terms of sparse matrix. In addition, we incorporate spatial regularization based on total variation (TV) and improve this way strongly robustness to noise. A landmark-based spectral clustering method is applied to the obtained sparse matrix, which further improves the clustering speed. Experimental results on two real HSIs demonstrate the effectiveness of the proposed method and the superior performance compared to both traditional SSC-based methods and the related large-scale clustering methods.

Sparse subspace clustering (SSC) has been widely applied in remote sensing demonstrating excellent performance. Recent extensions incorporate spatial information, typically via smoothness-enforcing regularization. We propose an alternative approach: a joint sparsity SSC model, where pixels within a local region are enforced to select a common set of samples in the subspace-sparse representation. The corresponding optimization problem is solved by the alternating direction method of multipliers (ADMM). Experimental results on real data show a significant improvement over SSC and related state-of-the-art methods.

Sparse subspace clustering (SSC) has achieved state‐of‐the‐art clustering quality by performing spectral clustering over an ℓ1‐norm based similarity graph. However, SSC is a transductive method, i.e. it cannot handle out‐of‐sample data that is not used to construct the graph. For each new datum, SSC requires solving n optimisation problems in O(n) variables, where n is the number of data points. Therefore, it is inefficient to apply SSC in fast online clustering and scalable grouping. An inductive spectral clustering algorithm called inductive SSC (iSSC) is proposed, which makes SSC feasible to cluster out‐of‐sample data. iSSC adopts the assumption that high‐dimensional data actually lie on the low‐dimensional manifold such that out‐of‐sample data could be grouped in the embedding space learned from in‐sample data. Experimental results show that iSSC is promising in clustering out‐of‐sample data.

Understanding the relationship between brain function and structure is vital important in the field of brain image analysis. It elucidates the working mechanism of the brain, which will contribute to better understand the brain and simulate the brain-like system. Extensive efforts have been made on this topic, but still far from the satisfactory. The major difficulties are at least two aspects. One is the huge individual difference among the subjects, which makes it hard to obtain stable results at groupwise level, e.g., noise signals can significantly affect the exploring process. The other one is the huge difference between functional and structural features of the brain, both in their pattern and size, which are very different. To alleviate the above problems, in this paper, we propose a two-stage multi-view low-rank sparse subspace clustering (Two-stage MLRSSC) method to jointly study the relationship between brain function and structure and identify the common regions of brain function and structure. The major innovation of proposed Two-stage MLRSSC is that comparable features of brain function and structure can be effectively extracted from low-rank sparse representation, and results are further improved the stability by two-stage strategy. Finally, groupwise-based stable functional and structural common regions are identified for better understanding the relationship. Experimental results shed new ways to explore the brain function and structure, new insights are observed and discussed.KeywordsBrain functionBrain structureMulti-ViewLow-Rank Sparse Subspace Clustering

Gene expression data is a kind of high dimension and small sample size data. The clustering accuracy of conventional clustering techniques is lower on gene expression data due to its high dimension. Because some subspace segmentation approaches can be better applied in the high-dimensional space, three new subspace clustering models for gene expression data sets are proposed in this work. The proposed projection subspace clustering models have projection sparse subspace clustering, projection low-rank representation subspace clustering and projection least-squares regression subspace clustering which combine projection technique with sparse subspace clustering, low-rank representation and least-square regression, respectively. In order to compute the inner product in the high-dimensional space, the kernel function is used to the projection subspace clustering models. The experimental results on six gene expression data sets show these models are effective.

Robust techniques such as sparse subspace clustering (SSC) have been recently developed for hyperspectral images (HSIs) based on the assumption that pixels belonging to the same land-cover class approximately lie in the same subspace. In order to account for the spatial information contained in HSIs, SSC models incorporating spatial information have become very popular. However, such models are often based on a local averaging constraint, which does not allow for a detailed exploration of the spatial information, thus limiting their discriminative capability and preventing the spatial homogeneity of the clustering results. To address these relevant issues, in this letter, we develop a new and effective $\ell _{2} $ -norm regularized SSC algorithm which adds a four-neighborhood $\ell _{2} $ -norm regularizer into the classical SSC model, thus taking full advantage of the spatial-spectral information contained in HSIs. The experimental results confirm the potential of including the spatial information (through the newly added $\ell _{2} $ -norm regularization term) in the SSC framework, which leads to a significant improvement in the clustering accuracy of SSC when applied to HSIs.

Compressive spectral imaging (CSI) acquires compressed observations of a spectral scene by applying different coding patterns at each spatial location and then performing a spectral-wise integration. Relying on compressive sensing, spectral image reconstruction is achieved by using nonlinear and relatively expensive optimization-based algorithms. In the CSI literature, several works have focused on improving reconstructions quality by properly designing the set of coding patterns. However, signal recovery is not actually necessary in many signal processing applications. For instance, assuming that compressed measurements with similar characteristics lie on the same subspace, unsupervised methods such as subspace clustering can be used to separate them into the same cluster. Since the structure of compressed measurements is defined by the applied codification, it is possible to improve clustering performance. This paper proposes to design a set of coding patterns such that inter-class and intra-class data structure is preserved after the CSI acquisition in order to improve clustering results directly on the compressed domain. To validate the coding pattern design, an algorithm based on sparse subspace clustering (SSC) is proposed to perform clustering on the compressed measurements. The proposed algorithm adds a three-dimensional (3-D) spatial regularizer to the SSC problem exploiting the spatial correlation of spectral images. In general, an overall accuracy up to 83.81% is obtained, when noisy measurements are assumed. In addition, a difference of at most 4% in terms of overall accuracy was observed when comparing the clustering results obtained by the full 3-D data with those achieved using CSI measurements acquired with the proposed coding pattern design.