Graph Regularized Low-Rank Representation for Semi-Supervised Learning

In the literature, most existing graph-based semi-supervised learning methods only use the label information of observed samples in the label propagation stage, while ignoring such valuable information when learning the graph. In this paper, we argue that it is beneficial to consider the label information in the graph learning stage. Specifically, by enforcing the weight of edges between labeled samples of different classes to be zero, we explicitly incorporate the label information into the state-of-the-art graph learning methods, such as the low-rank representation (LRR), and propose a novel semi-supervised graph learning method called semi-supervised low-rank representation. This results in a convex optimization problem with linear constraints, which can be solved by the linearized alternating direction method. Though we take LRR as an example, our proposed method is in fact very general and can be applied to any self-representation graph learning methods. Experiment results on both synthetic and real data sets demonstrate that the proposed graph learning method can better capture the global geometric structure of the data, and therefore is more effective for semi-supervised learning tasks.

Low Rank Representation (LRR) based unsupervised clustering methods have achieved great success since these methods could explore low-dimensional subspace structure embedded in original data effectively. The conventional LRR methods generally treat the data as the points in Euclidean space. However, it is no longer suitable for high-dimension data (such as video or imageset). That is because high-dimension data are always considered as non-linear manifold data such as Grassmann manifold. Besides, the typical LRR methods always adopt the traditional single nuclear norm based low rank constraint which can not fully reveal the low rank property of the data representation and often leads to suboptimal solution. In this paper, a new LRR based clustering model is constructed on Grassmann manifold for high-dimension data. In the proposed method, each high-dimension data is formed as a sample on Grassmann manifold with non-linear metric. Meanwhile, a non-convex low rank representation is adopt to reveal the intrinsic property of these high-dimension data and reweighted rank minimization constraint is introduced. The experimental results on several public datasets show that the proposed method outperforms the state-of-the-art clustering methods.

Symmetric low-rank representation with adaptive distance penalty for semi-supervised learning

Low-rank representation (LRR) has recently attracted a great deal of attention due to its pleasing efficacy in exploring low-dimensional subspace structures embedded in data. However, conventional LRR methods simply use Euclidean distance to measure the similarity of samples, ignoring the dynamic structure information of data. Meanwhile, recent studies have shown that a probabilistically motivated distance measurement (called effective distance) can model the dynamic structure information of data. In this paper, we propose an effective distance based LRR (EDLRR)method for representation learning. The proposed EDLRR method can not only represent the dynamic structure of data, but also capture the geometric information in the inherent nonlinear data. Our method mainly uses Effective Distance Computation and Effective Distance based Low-Rank Representation. We evaluate our method on datasets in the task of image classification, with results demonstrating the effectiveness of the method.

Data clustering is an important research topic in data mining and signal processing communications. In all the data clustering methods, the subspace spectral clustering methods based on self expression model, e.g., the Sparse Subspace Clustering (SSC) and the Low Rank Representation (LRR) methods, have attracted a lot of attention and shown good performance. The key step of SSC and LRR is to construct a proper affinity or similarity matrix of data for spectral clustering. Recently, Laplacian graph constraint was introduced into the basic SSC and LRR and obtained considerable improvement. However, the current graph construction methods do not well exploit and reveal the non-linear properties of the clustering data, which is common for high dimensional data. In this paper, we introduce the classic manifold learning method, the Local Linear Embedding (LLE), to learn the non-linear structure underlying the data and use the learned local geometry of manifold as a regularization for SSC and LRR, which results the proposed LLE-SSC and LLE-LRR clustering methods. Additionally, to solve the complex optimization problem involved in the proposed models, an efficient algorithm is also proposed. We test the proposed data clustering methods on several types of public databases. The experimental results show that our methods outperform typical subspace clustering methods with Laplacian graph constraint.

Low-rank representation with local constraint for graph construction

Multitask learning in distributed environments is an important problem in the field of federated learning. In many practical problems, human or machine experts try to solve different, but highly correlated tasks, thus their performance can be improved through collaboration, but their private datasets and model parameters cannot be shared with other participants because of business and privacy issues. Semi-supervised learning is also actively researched nowadays, which is motivated by the utilization of the usually free, large set of unlabeled data in concept and representation learning. These unlabeled samples can be utilized in the knowledge sharing of the participants during their collaboration. We propose a method, called FedLinked, which combines the capabilities of these area (federated, multitask and semi-supervised learning) related algorithms in the form of a privacy preserving complex solution-block targeting semi-supervised multitask cross-silo problems. FedLinked is a collaborative learning method, which is implemented through the regularization of representation learning based on the cross-utility of participating clients. We evaluate FedLinked on image classification problems and compare its performance to FedAvg based and non-cooperating clients based solutions in multitask scenarios. Besides, a detailed analysis of our proposed method is provided.

Recently, low-rank representation (LRR) methods have been widely applied for hyperspectral anomaly detection, due to their potentials in separating the backgrounds and anomalies. However, existing LRR models generally convert 3-D hyperspectral images (HSIs) into 2-D matrices, inevitably leading to the destruction of intrinsic 3-D structure properties in HSIs. To this end, we propose a novel tensor low-rank and sparse representation (TLRSR) method for hyperspectral anomaly detection. A 3-D TLR model is expanded to separate the LR background part represented by a tensorial background dictionary and corresponding coefficients. This representation characterizes the multiple subspace property of the complex LR background. Based on the weighted tensor nuclear norm and the LF,1 sparse norm, a dictionary is designed to make its atoms more relevant to the background. Moreover, a principal component analysis (PCA) method can be assigned as one preprocessing step to exact a subset of HSI bands, retaining enough the HSI object information and reducing computational time of the postprocessing tensorial operations. The proposed model is efficiently solved by the well-designed alternating direction method of multipliers (ADMMs). A comparison with the existing algorithms via experiments establishes the competitiveness of the proposed method with the state-of-the-art competitors in the hyperspectral anomaly detection task.

ABSTRACTIn the area of hyperspectral image (HSI) classification, graph-based semi-supervised learning (SSL) has been proved to be highly effective. Constructing a proper graph is critical for graph-based SSL tasks. In HSI, spectral distance is widely used to calculate the weight of graph edge, though it can be influenced by noise and outliers. Meanwhile, links among all the data points are incorporated in the graph, including those from different subspaces. Thus the constructed graph might contain incorrect information. In this letter, a novel semi-supervised HSI classification method using local low-rank representation (SL2R) is proposed. Edge weight calculation will not be affected by noise or outliers thanks to the robustness of low-rank representation (LRR). Since each graph is constructed at local level, where pixels are basically embedded in the same subspace, links among uncorrelated pixels can be removed. Moreover, spatial context is naturally characterized by low-rank constraint on adjacent pixels. Experimental results on two data sets (Indian Pines and Botswana) confirm the effectiveness of the proposed method.

A semi-supervised deep learning approach for cropped image detection

Supervised neighborhood graph construction for semi-supervised classification

Multiview subspace clustering is one of the most widely used methods for exploiting the internal structures of multiview data. Most previous studies have performed the task of learning multiview representations by individually constructing an affinity matrix for each view without simultaneously exploiting the intrinsic characteristics of multiview data. In this article, we propose a multiview low-rank representation (MLRR) method to comprehensively discover the correlation of multiview data for multiview subspace clustering. MLRR considers symmetric low-rank representations (LRRs) to be an approximately linear spatial transformation under the new base, that is, the multiview data themselves, to fully exploit the angular information of the principal directions of LRRs, which is adopted to construct an affinity matrix for multiview subspace clustering, under a symmetric condition. MLRR takes full advantage of LRR techniques and a diversity regularization term to exploit the diversity and consistency of multiple views, respectively, and this method simultaneously imposes a symmetry constraint on LRRs. Hence, the angular information of the principal directions of rows is consistent with that of columns in symmetric LRRs. The MLRR model can be efficiently calculated by solving a convex optimization problem. Moreover, we present an intuitive fusion strategy for symmetric LRRs from the perspective of spectral clustering to obtain a compact representation, which can be shared by multiple views and comprehensively represents the intrinsic features of multiview data. Finally, the experimental results based on benchmark datasets demonstrate the effectiveness and robustness of MLRR compared with several state-of-the-art multiview subspace clustering algorithms.

To overcome the limitations of existing low-rank representation (LRR) methods, i.e., the error distribution should be known a prior and the leading rank components might be over penalized, this paper proposes a new low-rank representation based model, namely double weighted LRR (DWLRR), using two distinguished properties on the concerned representation matrix. The first characterizes various distributions of the residuals into an adaptively learned weighting matrix for more flexibility of noise resistance. The second employs a parameterized rational penalty as well as a weighting vector s to reveal the importance of different rank components for better approximation to the intrinsic subspace structure. Moreover, we derive a computationally efficient algorithm based on the parallel updating scheme and automatic thresholding operation. Comprehensive experimental results conducted on image clustering demonstrate the robustness and efficiency of DWLRR compared with other state-of-the-art models.

With the development of cross-modal image fusion in multi-sensor systems, current fusion technologies have made significant progress in feature extraction, facilitating more effective image analysis. However, insufficient fusion information may degrade the correlation between the source and fused images, often resulting in the omission of critical features from the original modalities. Therefore, in order to preserve as much information as possible, especially for the complete extraction of effective feature information in source images, this paper proposes a new cross-modal image fusion method based on low-rank representation and convolutional sparse learning named LRCFuse. Firstly, the learned low-rank representation (LLRR) blocks are employed to perform dimensionality reduction on the source images while simultaneously extracting their low-rank and sparse feature components. Nevertheless, considering that the low-rank representation has insufficient modeling ability for different modal images, we introduce common feature preservation module (CFPM) blocks based on convolutional sparse coding. By leveraging the CFPM module, LRCFuse recovers common features from both source images to mitigate the loss caused by the imperfect assumptions of low-rank representation. Based on this, a multi-level optimization strategy incorporating pixel loss, shallow-level loss, mid-level loss, deep-level loss, and sobel loss is proposed to hierarchically learn and refine diverse image features. Quantitative and qualitative evaluations are conducted across various datasets, revealing that LRCFuse can effectively detect targets infrared salient targets, preserve additional details in visible images, and achieve better fusion results for subsequent downstream tasks.

In this paper, we study the problem of how to reliably compute neighborhoods on affinity graphs. The k-nearest neighbors (kNN) is one of the most fundamental and simple methods widely used in many tasks, such as classification and graph construction. Previous research focused on how to efficiently compute kNN on vectorial data. However, most real-world data have no vectorial representations, and only have affinity graphs which may contain unreliable affinities. Since the kNN of an object o is a set of k objects with the highest affinities to o, it is easily disturbed by errors in pairwise affinities between o and other objects, and also it cannot well preserve the structure underlying the data. To reliably analyze the neighborhood on affinity graphs, we define the k-dense neighborhood (kDN), which considers all pairwise affinities within the neighborhood, i.e., not only the affinities between o and its neighbors but also between the neighbors. For an object o, its kDN is a set kDN(o) of k objects which maximizes the sum of all pairwise affinities of objects in the set {o}?kDN(o). We analyze the properties of kDN, and propose an efficient algorithm to compute it. Both theoretic analysis and experimental results on shape retrieval, semi-supervised learning, point set matching and data clustering show that kDN significantly outperforms kNN on affinity graphs, especially when many pairwise affinities are unreliable.