Low-rank representation with adaptive graph regularization

Low-rank representation with local constraint for graph construction

Symmetric low-rank representation with adaptive distance penalty for semi-supervised learning

In the literature, most existing graph-based semi-supervised learning methods only use the label information of observed samples in the label propagation stage, while ignoring such valuable information when learning the graph. In this paper, we argue that it is beneficial to consider the label information in the graph learning stage. Specifically, by enforcing the weight of edges between labeled samples of different classes to be zero, we explicitly incorporate the label information into the state-of-the-art graph learning methods, such as the low-rank representation (LRR), and propose a novel semi-supervised graph learning method called semi-supervised low-rank representation. This results in a convex optimization problem with linear constraints, which can be solved by the linearized alternating direction method. Though we take LRR as an example, our proposed method is in fact very general and can be applied to any self-representation graph learning methods. Experiment results on both synthetic and real data sets demonstrate that the proposed graph learning method can better capture the global geometric structure of the data, and therefore is more effective for semi-supervised learning tasks.

Low Rank Representation (LRR) based unsupervised clustering methods have achieved great success since these methods could explore low-dimensional subspace structure embedded in original data effectively. The conventional LRR methods generally treat the data as the points in Euclidean space. However, it is no longer suitable for high-dimension data (such as video or imageset). That is because high-dimension data are always considered as non-linear manifold data such as Grassmann manifold. Besides, the typical LRR methods always adopt the traditional single nuclear norm based low rank constraint which can not fully reveal the low rank property of the data representation and often leads to suboptimal solution. In this paper, a new LRR based clustering model is constructed on Grassmann manifold for high-dimension data. In the proposed method, each high-dimension data is formed as a sample on Grassmann manifold with non-linear metric. Meanwhile, a non-convex low rank representation is adopt to reveal the intrinsic property of these high-dimension data and reweighted rank minimization constraint is introduced. The experimental results on several public datasets show that the proposed method outperforms the state-of-the-art clustering methods.

Data clustering is an important research topic in data mining and signal processing communications. In all the data clustering methods, the subspace spectral clustering methods based on self expression model, e.g., the Sparse Subspace Clustering (SSC) and the Low Rank Representation (LRR) methods, have attracted a lot of attention and shown good performance. The key step of SSC and LRR is to construct a proper affinity or similarity matrix of data for spectral clustering. Recently, Laplacian graph constraint was introduced into the basic SSC and LRR and obtained considerable improvement. However, the current graph construction methods do not well exploit and reveal the non-linear properties of the clustering data, which is common for high dimensional data. In this paper, we introduce the classic manifold learning method, the Local Linear Embedding (LLE), to learn the non-linear structure underlying the data and use the learned local geometry of manifold as a regularization for SSC and LRR, which results the proposed LLE-SSC and LLE-LRR clustering methods. Additionally, to solve the complex optimization problem involved in the proposed models, an efficient algorithm is also proposed. We test the proposed data clustering methods on several types of public databases. The experimental results show that our methods outperform typical subspace clustering methods with Laplacian graph constraint.

BackgroundIdentifying different types of cancer based on gene expression data has become hotspot in bioinformatics research. Clustering cancer gene expression data from multiple cancers to their own class is a significance solution. However, the characteristics of high-dimensional and small samples of gene expression data and the noise of the data make data mining and research difficult. Although there are many effective and feasible methods to deal with this problem, the possibility remains that these methods are flawed.ResultsIn this paper, we propose the graph regularized low-rank representation under symmetric and sparse constraints (sgLRR) method in which we introduce graph regularization based on manifold learning and symmetric sparse constraints into the traditional low-rank representation (LRR). For the sgLRR method, by means of symmetric constraint and sparse constraint, the effect of raw data noise on low-rank representation is alleviated. Further, sgLRR method preserves the important intrinsic local geometrical structures of the raw data by introducing graph regularization. We apply this method to cluster multi-cancer samples based on gene expression data, which improves the clustering quality. First, the gene expression data are decomposed by sgLRR method. And, a lowest rank representation matrix is obtained, which is symmetric and sparse. Then, an affinity matrix is constructed to perform the multi-cancer sample clustering by using a spectral clustering algorithm, i.e., normalized cuts (Ncuts). Finally, the multi-cancer samples clustering is completed.ConclusionsA series of comparative experiments demonstrate that the sgLRR method based on low rank representation has a great advantage and remarkable performance in the clustering of multi-cancer samples.

The conventional studies on different types of remote sensing (RS) images classifications are conducted separately. Thanks to the powerful potential of deep learning to automatically learn features from data, exploring a unified method is possible. Moreover, recent research shows that sparse and low-rank representations can convey valuable information for patterns classification. Therefore, this paper presents a two-stream heterogeneous RS images unified interpretation network (HRSIUI-Net). One stream is to transfer the pre-trained fully convolutional network to learn deep multi-scale spatial features of RS data. The other stream is to employ a subspace learning based on graph embedding to learn the sparse and low-rank subspace representations of high-dimensional features. And then, two streams of learned subspace features are integrated for classification combined with an SVM. The experimental results on two typical RS data indicate that HRSIUI-Net can achieve competitive performance.

Correlation learning is a technique utilized to find a common representation in cross-domain and multiview datasets. However, most existing methods are not robust enough to handle noisy data. As such, the common representation matrix learned could be influenced easily by noisy samples inherent in different instances of the data. In this paper, we propose a novel correlation learning method based on a low-rank representation, which learns a common representation between two instances of data in a latent subspace. Specifically, we begin by learning a low-rank representation matrix and an orthogonal rotation matrix to handle the noisy samples in one instance of the data so that a second instance of the data can linearly reconstruct the low-rank representation. Our method then finds a similarity matrix that approximates the common low-rank representation matrix much better such that a rank constraint on the Laplacian matrix would reveal the clustering structure explicitly without any spectral postprocessing. Extensive experimental results on ORL, Yale, Coil-20, Caltech 101-20, and UCI digits datasets demonstrate that our method has superior performance than other state-of-the-art compared methods in six evaluation metrics.

Discriminative feature extraction based on sparse and low-rank representation

Multiview subspace clustering is one of the most widely used methods for exploiting the internal structures of multiview data. Most previous studies have performed the task of learning multiview representations by individually constructing an affinity matrix for each view without simultaneously exploiting the intrinsic characteristics of multiview data. In this article, we propose a multiview low-rank representation (MLRR) method to comprehensively discover the correlation of multiview data for multiview subspace clustering. MLRR considers symmetric low-rank representations (LRRs) to be an approximately linear spatial transformation under the new base, that is, the multiview data themselves, to fully exploit the angular information of the principal directions of LRRs, which is adopted to construct an affinity matrix for multiview subspace clustering, under a symmetric condition. MLRR takes full advantage of LRR techniques and a diversity regularization term to exploit the diversity and consistency of multiple views, respectively, and this method simultaneously imposes a symmetry constraint on LRRs. Hence, the angular information of the principal directions of rows is consistent with that of columns in symmetric LRRs. The MLRR model can be efficiently calculated by solving a convex optimization problem. Moreover, we present an intuitive fusion strategy for symmetric LRRs from the perspective of spectral clustering to obtain a compact representation, which can be shared by multiple views and comprehensively represents the intrinsic features of multiview data. Finally, the experimental results based on benchmark datasets demonstrate the effectiveness and robustness of MLRR compared with several state-of-the-art multiview subspace clustering algorithms.

Constructing a powerful graph that can effectively depict the intrinsic connection of data points is the critical step to make the graph-based semisupervised learning algorithms achieve promising performance. Among popular graph construction algorithms, low-rank representation (LRR) is a very competitive one that can simultaneously explore the global structure of data and recover the data from noisy environments. Therefore, the learned low-rank coefficient matrix in LRR can be used to construct the data affinity matrix. Consider the existing problems such as the following: (1) the essentially linear property of LRR makes it not appropriate to process the possible nonlinear structure of data and (2) learning performance can be greatly enhanced by exploring the structure information of data; we propose a new manifold kernelized low-rank representation (MKLRR) model that can perform LRR in the data manifold adaptive kernel space. Specifically, the manifold structure can be incorporated into the kernel space by using graph Laplacian and thus the underlying geometry of data is reflected by the wrapped kernel space. Experimental results of semisupervised image classification tasks show the effectiveness of MKLRR. For example, MKLRR can, respectively, obtain 96.13%, 98.09%, and 96.08% accuracies on ORL, Extended Yale B, and PIE data sets when given 5, 20, and 20 labeled face images per subject.

Subspace clustering using a symmetric low-rank representation

Low-rank representation (LRR) has received considerable attention in subspace segmentation due to its effectiveness in exploring low-dimensional subspace structures embedded in data. To preserve the intrinsic geometrical structure of data, a graph regularizer has been introduced into LRR framework for learning the locality and similarity information within data. However, it is often the case that not only the high-dimensional data reside on a non-linear low-dimensional manifold in the ambient space, but also their features lie on a manifold in feature space. In this paper, we propose a dual graph regularized LRR model (DGLRR) by enforcing preservation of geometric information in both the ambient space and the feature space. The proposed method aims for simultaneously considering the geometric structures of the data manifold and the feature manifold. Furthermore, we extend the DGLRR model to include non-negative constraint, leading to a parts-based representation of data. Experiments are conducted on several image data sets to demonstrate that the proposed method outperforms the state-of-the-art approaches in image clustering.

Data clustering is a fundamental topic in machine learning and data mining areas. In recent years, researchers have proposed a series of effective methods based on Low Rank Representation (LRR) which could explore low-dimension subspace structure embedded in original data effectively. The traditional LRR methods are usually designed for vectorial data from linear spaces with Euclidean distance. However, high-dimension data (such as video clip or imageset) are always considered as non-linear manifold data such as Grassmann manifold with non-linear metric. In addition, traditional LRR clustering method always adopt single nuclear norm as low rank constraint which would lead to suboptimal solution and decrease the clustering accuracy. In this paper, we proposed a new low rank method on Grassmann manifold for video or imageset data clustering task. In the proposed method, video or imageset data are formulated as sample data on Grassmann manifold first. And then a double low rank constraint is proposed by combining the nuclear norm and bilinear representation to better construct the representation matrix. The experimental results on several public datasets show that the proposed method outperforms the state-of-the-art clustering methods.

Multi-view clustering can mine the underlying structure of multi-view data and has attracted increasing attention. Most existing multi-view clustering methods either construct the similarity matrix from the global level through self-representation learning or construct the similarity matrix from the local level through graph learning. Spectral clustering method can be used to yield the clustering results based on the similarity matrix. However, the similarity matrix that only considers global information or local information is not robust. Moreover, separating the similarity matrix learning and clustering as two steps may lead to sub-optimal clustering results. To address these issues, we propose in this paper, a multi-view subspace clustering with local and global information (MVSCLG) method. Our method combines the self-representation learning and graph learning to learn a similarity matrix with global and local information, and simultaneously utilizes the spectral decomposition and the spectral rotation techniques to yield the clustering results. We also develop an effective optimization algorithm to solve the resulting optimization problem. The effectiveness and superiority of this method are verified on four multi-view benchmark data sets.