Multiview Subspace Clustering Using Low-Rank Representation.

Multi-view subspace clustering, which aims to partition a dataset into its relevant subspaces based on their multi-view features, has been widely applied to identify various characteristics of datasets. The typical model of multi-view subspace clustering in literature often makes use of the nuclear norm to seek the underlying low-rank representation. However, due to the sum property of the singular values defined by tensor nuclear norm, the existing multi-view subspace clustering does not well handle the noise and the illumination variations embedded in multi-view data. To address and improve the robustness and clustering performance, we propose a new nonconvex multi-view subspace clustering model via tensor minimax concave penalty (MCP) approximation associated with rank minimization (NMSC-MCP), which can simultaneously construct the low-rank representation tensor and affinity matrix in a unified framework. Specifically, the nonconvex MCP approximation rank function is adopted to as a tighter tensor rank approximation to discriminate the dimension of features so that better accuracy can be achieved. In addition, we also address the local structure by including both hyper-Laplacian regularization and auto-weighting scheme into the objective function to promote the clustering performance. A corresponding iterative algorithm is then developed to solve the proposed model and the constructed iterative sequence generated by the proposed algorithm is shown to converge to the desirable KKT critical point. Extensive experiments on benchmark datasets have demonstrate the highly desirable effectiveness of our proposed method.

Multi-view subspace clustering has been an important and powerful tool for partitioning multi-view data, especially multi-view high-dimensional data. Despite great success, most of the existing multi-view subspace clustering methods still suffer from three limitations. First, they often recover the subspace structure in the original space, which can not guarantee the robustness when handling multi-view data with nonlinear structure. Second, these methods mostly regard subspace clustering and affinity matrix learning as two independent steps, which may not well discover the latent relationships among data samples. Third, many of them ignore the different importance of multiple views, whose performance may be badly affected by the low-quality views in multi-view data. To overcome these three limitations, this paper develops a novel subspace clustering method for multi-view data, termed Kernelized Multi-view Subspace Clustering via Auto-weighted Graph Learning (KMSC-AGL). Specifically, the proposed method implicitly maps the multi-view data from linear space into nonlinear space via kernel-induced functions, so as to exploit the nonlinear structure hidden in data. Furthermore, our method aims to enhance the clustering performance by learning a set of view-specific representations and their affinity matrix in a general framework. By integrating the view weighting strategy into this framework, our method can automatically assign the weights to different views, while learning an optimal affinity matrix that is well-adapted to the subsequent spectral clustering. Extensive experiments are conducted on a variety of multi-view data sets, which have demonstrated the superiority of the proposed method.

An increasing amount of attention have been attracted in multi-view subspace clustering, whose impressive performance can be achieved by means of the self-expressive property, under the assumption of linear relations between multi-view data samples. However, most of them fail to recover the nonlinear relations between multi-view data for deeper study, and additionally they are incapable of discovering the comprehensiveness and higher-order correlations among multiple views. To deal with these challenges, we propose a novel model termed Coupled Learning for Kernel Representation and Graph Tensor (CLKT) in Multi-view Subspace Clustering, where both nonlinear relations and higher-order correlations among multiple affinity graphs are jointly learned in a unified framework. Optimal solutions of the proposed method can be obtained by an alternative minimizing optimization strategy. Extensive experiments on six real-world datasets indicate the superiority of CLKT compared with the state-of-the-art methods.KeywordsMulti-view clusteringKernel learningLow-rank tensor representation

Data clustering is an important research topic in data mining and signal processing communications. In all the data clustering methods, the subspace spectral clustering methods based on self expression model, e.g., the Sparse Subspace Clustering (SSC) and the Low Rank Representation (LRR) methods, have attracted a lot of attention and shown good performance. The key step of SSC and LRR is to construct a proper affinity or similarity matrix of data for spectral clustering. Recently, Laplacian graph constraint was introduced into the basic SSC and LRR and obtained considerable improvement. However, the current graph construction methods do not well exploit and reveal the non-linear properties of the clustering data, which is common for high dimensional data. In this paper, we introduce the classic manifold learning method, the Local Linear Embedding (LLE), to learn the non-linear structure underlying the data and use the learned local geometry of manifold as a regularization for SSC and LRR, which results the proposed LLE-SSC and LLE-LRR clustering methods. Additionally, to solve the complex optimization problem involved in the proposed models, an efficient algorithm is also proposed. We test the proposed data clustering methods on several types of public databases. The experimental results show that our methods outperform typical subspace clustering methods with Laplacian graph constraint.

Multi-view clustering can mine the underlying structure of multi-view data and has attracted increasing attention. Most existing multi-view clustering methods either construct the similarity matrix from the global level through self-representation learning or construct the similarity matrix from the local level through graph learning. Spectral clustering method can be used to yield the clustering results based on the similarity matrix. However, the similarity matrix that only considers global information or local information is not robust. Moreover, separating the similarity matrix learning and clustering as two steps may lead to sub-optimal clustering results. To address these issues, we propose in this paper, a multi-view subspace clustering with local and global information (MVSCLG) method. Our method combines the self-representation learning and graph learning to learn a similarity matrix with global and local information, and simultaneously utilizes the spectral decomposition and the spectral rotation techniques to yield the clustering results. We also develop an effective optimization algorithm to solve the resulting optimization problem. The effectiveness and superiority of this method are verified on four multi-view benchmark data sets.

Subspace clustering using a symmetric low-rank representation

Recently, low-rank representation (LRR) methods have been widely applied for hyperspectral anomaly detection, due to their potentials in separating the backgrounds and anomalies. However, existing LRR models generally convert 3-D hyperspectral images (HSIs) into 2-D matrices, inevitably leading to the destruction of intrinsic 3-D structure properties in HSIs. To this end, we propose a novel tensor low-rank and sparse representation (TLRSR) method for hyperspectral anomaly detection. A 3-D TLR model is expanded to separate the LR background part represented by a tensorial background dictionary and corresponding coefficients. This representation characterizes the multiple subspace property of the complex LR background. Based on the weighted tensor nuclear norm and the LF,1 sparse norm, a dictionary is designed to make its atoms more relevant to the background. Moreover, a principal component analysis (PCA) method can be assigned as one preprocessing step to exact a subset of HSI bands, retaining enough the HSI object information and reducing computational time of the postprocessing tensorial operations. The proposed model is efficiently solved by the well-designed alternating direction method of multipliers (ADMMs). A comparison with the existing algorithms via experiments establishes the competitiveness of the proposed method with the state-of-the-art competitors in the hyperspectral anomaly detection task.

Low-rank representation (LRR) has recently attracted a great deal of attention due to its pleasing efficacy in exploring low-dimensional subspace structures embedded in data. However, conventional LRR methods simply use Euclidean distance to measure the similarity of samples, ignoring the dynamic structure information of data. Meanwhile, recent studies have shown that a probabilistically motivated distance measurement (called effective distance) can model the dynamic structure information of data. In this paper, we propose an effective distance based LRR (EDLRR)method for representation learning. The proposed EDLRR method can not only represent the dynamic structure of data, but also capture the geometric information in the inherent nonlinear data. Our method mainly uses Effective Distance Computation and Effective Distance based Low-Rank Representation. We evaluate our method on datasets in the task of image classification, with results demonstrating the effectiveness of the method.

Biomedical cell image contains different types of components. The segmentation of these components is essential for characterizing the overall structure of leucocytes. However, the task is challenging because the data generally possess irregular patterns, varying eccentricity and congestion of inter class substructures. In order to perform the automatic segmentation of leucocytes, many approaches in machine learning have been proposed. Most of the recent methods utilize k-means and fuzzy c-means algorithms. These methods suffer from the under-segmentation and the oversegmentation of data points. We have adopted an efficient Low Rank Representation (LRR) method for the segmentation of cell image patterns. Its overall segmentation performance is very high in comparison to the k-means and fuzzy c-means segmentation methods. It discriminates sub structures by incorporating multiple feature data into the joint representation matrix. The main improvements of our adopted approach, include the global geometric representation of different cell patterns and the control of different sources of noise in the data. Data points of the cell image are evolved with the basic assumption as x = m + n. Where x is the input observation, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$m$</tex> is the set of data points that lies in the correct subspace and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex> represent the noise in the data. The performance of the model was evaluated on the different types of medical images. It has reported the clustering accuracy of 97.98% on the biomedical database. Whereas an accuracy of 93.1 % was reported on ALL-IDB cell dataset.

BackgroundIdentifying different types of cancer based on gene expression data has become hotspot in bioinformatics research. Clustering cancer gene expression data from multiple cancers to their own class is a significance solution. However, the characteristics of high-dimensional and small samples of gene expression data and the noise of the data make data mining and research difficult. Although there are many effective and feasible methods to deal with this problem, the possibility remains that these methods are flawed.ResultsIn this paper, we propose the graph regularized low-rank representation under symmetric and sparse constraints (sgLRR) method in which we introduce graph regularization based on manifold learning and symmetric sparse constraints into the traditional low-rank representation (LRR). For the sgLRR method, by means of symmetric constraint and sparse constraint, the effect of raw data noise on low-rank representation is alleviated. Further, sgLRR method preserves the important intrinsic local geometrical structures of the raw data by introducing graph regularization. We apply this method to cluster multi-cancer samples based on gene expression data, which improves the clustering quality. First, the gene expression data are decomposed by sgLRR method. And, a lowest rank representation matrix is obtained, which is symmetric and sparse. Then, an affinity matrix is constructed to perform the multi-cancer sample clustering by using a spectral clustering algorithm, i.e., normalized cuts (Ncuts). Finally, the multi-cancer samples clustering is completed.ConclusionsA series of comparative experiments demonstrate that the sgLRR method based on low rank representation has a great advantage and remarkable performance in the clustering of multi-cancer samples.

Multi-view data clustering often aims to utilize various representations or views of original data to improve the clustering performance compared to the single-view clustering approach. Most multi-view subspace clustering methods are proposed to construct the affinity matrix of each view individually and then implement with spectral clustering for multi-view data clustering. The multi-view low-rank sparse subspace clustering (MLRSSC) is an effective and popular clustering algorithm among multi-view subspace clustering. This method can explore the joint subspace representation through creating an affinity matrix integrated of all views of input data. In addition, the low-rank and sparsity constraints are introduced into this method to enhance the clustering results. However, the original MLRSSC uses the mean square error as the fidelity term while not consider the complex noise pollution in real situations. Therefore, we introduce a complex noise modeling approach, i.e., independent and piecewise identically distributed (IPID) noise model, for MLRSSC to improve its performance. The related experimental results confirm that this proposed algorithm surpasses many state-of-the-art subspace clustering methods on several real-world datasets.

Low Rank Representation (LRR) based unsupervised clustering methods have achieved great success since these methods could explore low-dimensional subspace structure embedded in original data effectively. The conventional LRR methods generally treat the data as the points in Euclidean space. However, it is no longer suitable for high-dimension data (such as video or imageset). That is because high-dimension data are always considered as non-linear manifold data such as Grassmann manifold. Besides, the typical LRR methods always adopt the traditional single nuclear norm based low rank constraint which can not fully reveal the low rank property of the data representation and often leads to suboptimal solution. In this paper, a new LRR based clustering model is constructed on Grassmann manifold for high-dimension data. In the proposed method, each high-dimension data is formed as a sample on Grassmann manifold with non-linear metric. Meanwhile, a non-convex low rank representation is adopt to reveal the intrinsic property of these high-dimension data and reweighted rank minimization constraint is introduced. The experimental results on several public datasets show that the proposed method outperforms the state-of-the-art clustering methods.

Symmetric low-rank representation with adaptive distance penalty for semi-supervised learning

Subspace clustering based on latent low rank representation with Frobenius norm minimization

This paper addresses the multi-view subspace clustering problem and proposes the self-paced enhanced low-rank tensor kernelized multi-view subspace clustering (SETKMC) method, which is based on two motivations: (1) singular values of the representations and multiple instances should be treated differently. The reasons are that larger singular values of the representations usually quantify the major information and should be less penalized; samples with different degrees of noise may have various reliability for clustering. (2) many existing methods may cause the degraded performance when multi-view features reside in different nonlinear subspaces. This is because they usually assumed that multiple features lie within the union of several linear subspaces. SETKMC integrates the nonconvex tensor norm, self-paced learning, and kernel trick into a unified model for multi-view subspace clustering. The nonconvex tensor norm imposes different weights on different singular values. The self-paced learning gradually involves instances from more reliable to less reliable ones while the kernel trick aims to handle the multi-view data in nonlinear subspaces. One iterative algorithm is proposed based on the alternating direction method of multipliers. Extensive results on seven real-world datasets show the effectiveness of the proposed SETKMC compared to fifteen state-of-the-art multi-view clustering methods.