Dual Graph Regularized Latent Low-Rank Representation for Subspace Clustering.

BackgroundIdentifying different types of cancer based on gene expression data has become hotspot in bioinformatics research. Clustering cancer gene expression data from multiple cancers to their own class is a significance solution. However, the characteristics of high-dimensional and small samples of gene expression data and the noise of the data make data mining and research difficult. Although there are many effective and feasible methods to deal with this problem, the possibility remains that these methods are flawed.ResultsIn this paper, we propose the graph regularized low-rank representation under symmetric and sparse constraints (sgLRR) method in which we introduce graph regularization based on manifold learning and symmetric sparse constraints into the traditional low-rank representation (LRR). For the sgLRR method, by means of symmetric constraint and sparse constraint, the effect of raw data noise on low-rank representation is alleviated. Further, sgLRR method preserves the important intrinsic local geometrical structures of the raw data by introducing graph regularization. We apply this method to cluster multi-cancer samples based on gene expression data, which improves the clustering quality. First, the gene expression data are decomposed by sgLRR method. And, a lowest rank representation matrix is obtained, which is symmetric and sparse. Then, an affinity matrix is constructed to perform the multi-cancer sample clustering by using a spectral clustering algorithm, i.e., normalized cuts (Ncuts). Finally, the multi-cancer samples clustering is completed.ConclusionsA series of comparative experiments demonstrate that the sgLRR method based on low rank representation has a great advantage and remarkable performance in the clustering of multi-cancer samples.

In the fields of machine learning and data mining, label learning is a nascent area of research, and within this paradigm, there is much room for improving multi-label manifold learning algorithms for high-dimensional data. Thus far, researchers have experimented with mapping relationships from the feature space to the traditional logical label space (using neighbors in the label space, for example, to predict logical label vectors from the feature space's manifold structure). Here we combine the feature manifold's and label space's local topological structures to reconstruct the label manifold. To achieve this, we use a nonlinear manifold learning algorithm to transform the local topological structure from the feature space to the label space. Our algorithm adopts a regularized least-squares kernel method to realize the reconstruction process, employing an optimization function to find the best solution. Extensive experiments show that our algorithm significantly improves multi-label manifold learning in terms of learning accuracy and time complexity.

Subspace clustering based on latent low rank representation with Frobenius norm minimization

Non-negative Matrix Factorization (NMF) is widely used in image clustering; however, it has inherent limitations, including its unsupervised nature, lack of sparsity constraints, inability to leverage label information, and difficulty in capturing the geometric structure and nonlinear characteristics of data. To address these limitations, this paper proposes a Sparse Kernel Graph-regularized Discriminant Non-negative Matrix Factorization (SKGDNMF) algorithm. The algorithm innovatively adopts a dual normalization strategy, which involves column normalization for the basis matrix and row normalization for the coefficient matrix, and integrates spectral clustering to construct an end-to-end deep clustering framework. By applying sparse regularization to the coefficient matrix, the model’s robustness is significantly improved; this regularization forms complementary optimization with graph regularization, thereby effectively alleviating overfitting. A three-dimensional golden-section parameter optimization method is employed to determine key parameters, which enhances the algorithm’s practicality. Comparative experiments conducted on multiple datasets show that SKGDNMF significantly outperforms mainstream algorithms in terms of sparsity, robustness, and clustering performance, indicating its superior effectiveness for image clustering tasks.

Feature extraction (FE) methods based on low-rank representation (LRR) have become important topics in hyperspectral images (HSIs) data analysis. In this paper, a supervised FE method for HSIs data based on LRR with the ability to preserve the local pairwise constraints information (LRLPC) is proposed. LRLPC does not change the data dimensionality and only employs a technique to enrich the original feature space (OFS) and to obtain enriched feature space, which results in features richer than OFS. To overcome the problem of LRR in lacking the local structure information (LSI) of data, a local discriminative regularization term is imposed on the fitness function of LRR to keep the LSI of data. For nonlinear structure of data, LRLPC is extended to kernel LRLPC (KLRLPC) using kernel trick. Utilization of existing information in the pairwise constraints is useful for limited labeled samples situations as a common problem in HSI data analysis. The obtained experimental results using two well-known HSI data sets confirm the effectiveness of LRLPC and KLRLPC for dimension reduction and classification of HSIs.

Local and global regularized sparse coding for data representation

Hyperspectral unmixing, aiming to estimate the fractional abundances of pure spectral signatures in each mixed pixel, has attracted considerable attention in analyzing hyperspectral images. Plenty of sparse unmixing methods have been proposed in the literature that achieved promising performance. However, many of these methods overlook the latent geometrical structure of the hyperspectral data which limit their performance to some extent. To address this issue, a double reweighted sparse and graph regularized unmixing method is proposed in this paper. Specifically, a graph regularizer is employed to capture the correlation information between abundance vectors, which makes use of the property that similar pixels in a spectral neighborhood have higher probability to share similar abundances. In this way, the latent geometrical structure of the hyperspectral data can be transferred to the abundance space. In addition, a double weighted sparse regularizer is used to enhance the sparsity of endmembers and the fractional abundance maps, where one weight is introduced to promote the sparsity of endmembers as a hyperspectral image typically contains fewer endmembers compared to the overcomplete spectral library and the other weight is exploited to improve the sparsity of the abundance matrix. The weights of the double weighted sparse regularizer used for the next iteration are adaptively computed from the current abundance matrix. The experimental results on synthetic and real hyperspectral data demonstrate the superiority of our method compared with some state-of-the-art approaches.

Constructing a powerful graph that can effectively depict the intrinsic connection of data points is the critical step to make the graph-based semisupervised learning algorithms achieve promising performance. Among popular graph construction algorithms, low-rank representation (LRR) is a very competitive one that can simultaneously explore the global structure of data and recover the data from noisy environments. Therefore, the learned low-rank coefficient matrix in LRR can be used to construct the data affinity matrix. Consider the existing problems such as the following: (1) the essentially linear property of LRR makes it not appropriate to process the possible nonlinear structure of data and (2) learning performance can be greatly enhanced by exploring the structure information of data; we propose a new manifold kernelized low-rank representation (MKLRR) model that can perform LRR in the data manifold adaptive kernel space. Specifically, the manifold structure can be incorporated into the kernel space by using graph Laplacian and thus the underlying geometry of data is reflected by the wrapped kernel space. Experimental results of semisupervised image classification tasks show the effectiveness of MKLRR. For example, MKLRR can, respectively, obtain 96.13%, 98.09%, and 96.08% accuracies on ORL, Extended Yale B, and PIE data sets when given 5, 20, and 20 labeled face images per subject.

Embedded conformal deep low-rank auto-encoder network for matrix recovery

Benefiting from the joint consideration of geometric structures and low-rank constraint, graph low-rank representation (GLRR) method has led to the state-of-the-art results in many applications. However, it faces the limitations that the structure of errors should be known a prior, the isolated construction of graph Laplacian matrix, and the over shrinkage of the leading rank components. To improve GLRR in these regards, this paper proposes a new LRR model, namely iterative reconstrained LRR via weighted nonconvex regularization, using three distinguished properties on the concerned representation matrix. The first characterizes various distributions of the errors into an adaptively learned weight factor for more flexibility of noise suppression. The second generates an accurate graph matrix from weighted observations for less afflicted by noisy features. The third employs a parameterized rational function to reveal the importance of different rank components for better approximation to the intrinsic subspace structure. Following a deep exploration of automatic thresholding, parallel update, and partial SVD operation, we derive a computationally efficient low-rank representation algorithm using an iterative reconstrained framework and accelerated proximal gradient method. Comprehensive experiments are conducted on synthetic data, image clustering, and background subtraction to achieve several quantitative benchmarks as clustering accuracy, normalized mutual information, and execution time. Results demonstrate the robustness and efficiency of IRWNR compared with other state-of-the-art models.

In this paper, we propose a new image clustering algorithm, referred to as clustering using local discriminant models and global integration (LDMGI). To deal with the data points sampled from a nonlinear manifold, for each data point, we construct a local clique comprising this data point and its neighboring data points. Inspired by the Fisher criterion, we use a local discriminant model for each local clique to evaluate the clustering performance of samples within the local clique. To obtain the clustering result, we further propose a unified objective function to globally integrate the local models of all the local cliques. With the unified objective function, spectral relaxation and spectral rotation are used to obtain the binary cluster indicator matrix for all the samples. We show that LDMGI shares a similar objective function with the spectral clustering (SC) algorithms, e.g., normalized cut (NCut). In contrast to NCut in which the Laplacian matrix is directly calculated based upon a Gaussian function, a new Laplacian matrix is learnt in LDMGI by exploiting both manifold structure and local discriminant information. We also prove that K-means and discriminative K-means (DisKmeans) are both special cases of LDMGI. Extensive experiments on several benchmark image datasets demonstrate the effectiveness of LDMGI. We observe in the experiments that LDMGI is more robust to algorithmic parameter, when compared with NCut. Thus, LDMGI is more appealing for the real image clustering applications in which the ground truth is generally not available for tuning algorithmic parameters.

Classical linear regression not only lacks of the flexibility in fitting the label, but also ignores to preserve the intrinsic local geometric structure of data, which leads to overfitting. In this paper, we propose a novel discriminative regression method, called adaptive locality preserving based discriminative regression (ALPDR), to address these problems. Firstly, a locality preserving constraint regularized by the adaptive weight is introduced to preserve the intrinsic geometric structures of data, in which the similar points of the same class are adaptively pulled together by the projection. Secondly, ALPDR directly learns the discriminative target matrix from data based on the given label information, which allows more freedom in label fitting and simultaneously enlarges the margins between different classes. Thirdly, ALPDR imposes a row-sparsity constraint on the projection, which enables the method to adaptively select the most discriminative features from data such that the negative influence of noises and redundant features can be eliminated. Finally, an efficient iterative algorithm is provided to optimize the model. Extensive experiments show that the proposed method outperforms the other state-of-art methods, which proves the effectiveness of the proposed method.

In multimedia applications, dimension reduction is essential to the effectiveness and efficiency of an algorithm due to the curse of dimensionality. Recently, its adaptive variants have received considerable attention in unsupervised learning since a single pass without label information often fails to guarantee an optimal representation, especially when the parameters are not set properly. However, most such methods are basically linear, therefore unable to consider the geometrical structure of the data space. In this paper, we propose a novel algorithm called Nonlinear Adaptive Dimension Reduction (NADR), which adaptively learns the optimal low-dimensional coordinates that preserve the intrinsic geometric structure of the original data. Moreover, the incorporation of K-means enables NADR to be a powerful alternative for cluster analysis. Experiments on benchmark image data sets illustrate that NADR outperforms the state-of-the-art adaptive dimension reduction methods.

Nonnegative low-rank representation based manifold embedding for semi-supervised learning

In this paper, a noise robust tensor based image clustering approach is proposed which can also perform well in the presence of gross errors. Our major contribution is the improved submodule identification technique in noisy environment by incorporating three important improvements: better low rank representation using reweighted nuclear norm, $$l_\frac{1}{2}$$ regularization to accurately capture sparseness and an error term in the model for noise robustness. Reweighted nuclear norm is introduced in the clustering model to capture self-expressiveness property in a better manner. The $$l_\frac{1}{2}$$ norm regularization is applied in place of $$l_1$$ -norm to properly capture the correlation among data members. An error term is introduced into the model to separate noise and data, which brings a noise robust image clustering technique. Combined effect all the three factors results an accurate clustering method even under the presence of severe noise. The performance of the proposed method is tested on different datasets with varying amount of noise. It is found that the proposed method provides better classification accuracy in almost all conditions as compared with existing methods.