Graph based semi-supervised classification with probabilistic nearest neighbors

Graph-based semi-supervised learning (SSL) algorithms have been widely studied in the last few years. Most of these algorithms were designed from unconstrained optimization problems using a Laplacian regularizer term as smoothness functional in an attempt to reflect the intrinsic geometric structure of the data's marginal distribution. Although a number of recent research papers are still focusing on unconstrained methods for graph-based SSL, a recent statistical analysis showed that many of these algorithms may be unstable on transductive regression. Therefore, we focus on providing new constrained methods for graph-based SSL. We begin by analyzing the regularization framework of existing unconstrained methods. Then, we incorporate two normalization constraints into the optimization problem of three of these methods. We show that the proposed optimization problems have closed-form solution. By generalizing one of these constraints to any distribution, we provide generalized methods for constrained graph-based SSL. The proposed methods have a more flexible regularization framework than the corresponding unconstrained methods. More precisely, our methods can deal with any graph Laplacian and use higher order regularization, which is effective on general SSL taks. In order to show the effectiveness of the proposed methods, we provide comprehensive experimental analyses. Specifically, our experiments are subdivided into two parts. In the first part, we evaluate existing graph-based SSL algorithms on time series data to find their weaknesses. In the second part, we evaluate the proposed constrained methods against six state-of-the-art graph-based SSL algorithms on benchmark data sets. Since the widely used best case analysis may hide useful information concerning the SSL algorithms' performance with respect to parameter selection, we used recently proposed empirical evaluation models to evaluate our results. Our results show that our methods outperforms the competing methods on most parameter settings and graph construction methods. However, we found a few experimental settings in which our methods showed poor performance. In order to facilitate the reproduction of our results, the source codes, data sets, and experimental results are freely available.

The key problem of Graph-Based Semi-Supervised Learning (GBSSL) methods is how to construct the graph structure under some assumptions. While distance information among graph nodes is investigated well for graph construction, the density information is not given enough attention. In this paper, we propose a novel GBSSL method, named Density-Sensitive Manifold Learning (DSML), which introduces density distribution into graph construction by calculating a new propagation coefficient matrix. The experimental results show that DSML scheme performs better than traditional GBSSL methods. More importantly, the new propagation coefficient matrix can be easily introduced into traditional GBSSL methods to improve their performance, which is also validated in the experiments.

Semi-supervised learning with a graph-based approach has gained prominence in machine learning, particularly in scenarios where labeling data involves substantial costs. Graph convolution networks (GCNs) have found widespread application in semi-supervised learning, predominantly on graph-structured data such as citation and social networks. However, a noticeable gap exists in the application of these methods to non-graph multi-view data, such as collections of images. In an effort to address this gap, we introduce two innovative deep semi-supervised multi-view classification models specifically tailored for non-graph data. Both models share a common architecture, leveraging GCNs and integrating a label smoothing constraint. The primary distinction lies in the construction of the consensus similarity graph. The first model directly reconstructs the consensus graph from different views using a specialized objective function designed for flexible graph-based semi-supervised classification. In contrast, the second model independently reconstructs individual graphs and subsequently adaptively merges them into a unified consensus graph. Our experiments encompass various multiple-view image datasets. The results consistently demonstrate the superior performance of our proposed approach compared to traditional fusion methods with GCNs. In this research, we present two approaches for tackling semi-supervised classification challenges involving multiple views. One method is named Semi-supervised Classification with a Unified Graph (SCUG), and the other is referred to as Semi-supervised Classification with a Fused Graph (SC-Fused). Both methods share a common semi-supervised classification process, utilizing the GCN framework and incorporating label smoothing. However, the key distinction lies in the construction of the similarity graph. Unlike traditional ad hoc graph construction approaches, our proposed methods, SCUG and SC-Fused, estimate the unified graph or individual graphs, respectively, alongside the labels. This results in more optimized graphs that benefit from data smoothing and the semi-supervised context.

Sample-weighted fused graph-based semi-supervised learning on multi-view data

Semi-supervised learning through adaptive Laplacian graph trimming

An outstanding problem in reconstruction methodology is the treatment of incomplete data sets. The inexact reconstruction technique (IRT) allows a stochastic approach, carried out in real space, which provides substantial improvement in reconstruction accuracy when compared to the standard filtered backprojection algorithm. This technique relies specifically on an iterative approach to the treatment of the probability related matrix which is assumed to be proportional to the original density distribution (original object matrix). This is obtained by the summation or multiplication of the set of probability matrices generated by back-projecting the known projections after the usual reorientation to correct for the angular projection to which each corresponds. Employing a Boolean constraint, the largest value in the probability matrix is located and an `l' is placed at the same coordinates in a blank array. This point in the probability matrix is then set to zero and the matrix projections are taken at the same angular orientations followed by re-backprojection to generate a new probability matrix. This process is repeated until the probability matrix is depleted or a specified mass is reached in the reconstructed object. The algorithm requires a considerable amount of computer time due to the necessity of recreating the probability matrix after each point is taken. A compromise solution is to address a certain fraction of the probability matrix to reduce the number of iterations. It has been demonstrated that reasonable results can be obtained when the probability matrix is addressed in increments of 5% or less. In this paper we demonstrate the use of an IRT on the reconstruction of multiple grey level images using limited data sets of from four to sixteen projections.

Semi-Supervised Learning (SSL) techniques have become very relevant since they require a small set of labeled data. In this context, graph-based algorithms have gained prominence in the area due to their capacity to exploiting, besides information about data points, the relationships among them. Moreover, data represented in graphs allow the use of collective inference (vertices can affect each other), propagation of labels (autocorrelation among neighbors) and use of neighborhood characteristics of a vertex. An important step in graph-based SSL methods is the conversion of tabular data into a weighted graph. The graph construction has a key role in the quality of the classification in graph-based methods. This paper explores a method for graph construction that uses available labeled data. We provide extensive experiments showing the proposed method has many advantages: good classification accuracy, quadratic time complexity, no sensitivity to the parameter k > 10, sparse graph formation with average degree around 2 and hub formation from the labeled points, which facilitates the propagation of labels.

ABSTRACTIn the area of hyperspectral image (HSI) classification, graph-based semi-supervised learning (SSL) has been proved to be highly effective. Constructing a proper graph is critical for graph-based SSL tasks. In HSI, spectral distance is widely used to calculate the weight of graph edge, though it can be influenced by noise and outliers. Meanwhile, links among all the data points are incorporated in the graph, including those from different subspaces. Thus the constructed graph might contain incorrect information. In this letter, a novel semi-supervised HSI classification method using local low-rank representation (SL2R) is proposed. Edge weight calculation will not be affected by noise or outliers thanks to the robustness of low-rank representation (LRR). Since each graph is constructed at local level, where pixels are basically embedded in the same subspace, links among uncorrelated pixels can be removed. Moreover, spatial context is naturally characterized by low-rank constraint on adjacent pixels. Experimental results on two data sets (Indian Pines and Botswana) confirm the effectiveness of the proposed method.

Graph-based semisupervised learning is of great importance in many effective learning systems, particularly in agnostic settings where no parametric information or other prior knowledge about the data distribution is available. It leverages the graph structure to propagate labels from labeled nodes to unlabeled ones. Two separate stages are usually involved: constructing an affinity graph and propagating labels on the graph for transductive inference. It is suboptimal to manage them independently, as the correlation between the affinity graph and the labels would not be fully exploited. In this article, we integrate these two stages into one unified framework by formulating the graph construction as a regularized function estimation problem, similar to label propagation. We then propose an alternating diffusion process to solve them alternately, which allows us to learn the graph and unknown labels in an iterative fashion. With the proposed framework, we can construct a dynamic graph adapted to the given and predicted labels iteratively, resulting in more accurate and robust label propagation performance. Extensive experiments on synthetic data and various real-world data have demonstrated the superiority of the proposed method compared with other state-of-the-art methods.

Graph-based semi-supervised learning has recently come into focus for to its two defining phases: graph construction, which converts the data into a graph, and label inference, which predicts the appropriate labels for unlabeled data using the constructed graph. And the label inference is based on the smoothness assumption of semi-supervised learning. In this study, we propose an enhanced label inference approach which incorporates the importance of each vertex into the existing inference algorithms to improve the prediction capabilities of the algorithms. We also present extensions of three algorithms which are capable of taking the vertex importance variable to apply in learning. Experiments show that our algorithms perform better than the base algorithms on a variety of datasets, especially when the data is less smooth over the graphs.

The recent years have witnessed a surge of interests in graph-based semi-supervised learning (GBSSL). In this paper, we will introduce a series of works done by our group on this topic including: 1) a method called linear neighborhood propagation (LNP) which can automatically construct the optimal graph; 2) a novel multilevel scheme to make our algorithm scalable for large data sets; 3) a generalized point charge scheme for GBSSL; 4) a multilabel GBSSL method by solving a Sylvester equation; 5) an information fusion framework for GBSSL; and 6) an application of GBSSL on fMRI image segmentation.

Local learning integrating global structure for large scale semi-supervised classification

Graph neural network (GNN) and label propagation algorithm (LPA) are both message passing algorithms, which have achieved superior performance in semi-supervised classification. GNN performs feature propagation by a neural network to make predictions, while LPA uses label propagation across graph adjacency matrix to get results. However, there is still no effective way to directly combine these two kinds of algorithms. To address this issue, we propose a novel Unified Message Passaging Model (UniMP) that can incorporate feature and label propagation at both training and inference time. First, UniMP adopts a Graph Transformer network, taking feature embedding and label embedding as input information for propagation. Second, to train the network without overfitting in self-loop input label information, UniMP introduces a masked label prediction strategy, in which some percentage of input label information are masked at random, and then predicted. UniMP conceptually unifies feature propagation and label propagation and is empirically powerful. It obtains new state-of-the-art semi-supervised classification results in Open Graph Benchmark (OGB).

In this paper, we propose a novel graph-based semisupervised learning framework, called joint sparse representation and embedding propagation learning (JSREPL). The idea of JSREPL is to join EPL with sparse representation to perform label propagation. Like most of graph-based semisupervised propagation learning algorithms, JSREPL also constructs weights graph matrix from given data. Different from classical approaches which build weights graph matrix and estimate the labels of unlabeled data in sequence, JSREPL simultaneously builds weights graph matrix and estimates the labels of unlabeled data. We also propose an efficient algorithm to solve the proposed problem. The proposed method is applied to the problem of semisupervised image clustering using the ORL, Yale, PIE, and YaleB data sets. Our experiments demonstrate the effectiveness of our proposed algorithm.

In real worlds applications, some former research papers have shown that manifold learning tries to discover the non-linear low-dimensional data manifold from a high-dimensional space. Many natural images and the face images are believed to be sampled from a manifold. In this paper, we try to investigate whether discovering such manifold can aid the semi-supervised learning algorithms. We propose a novel graph-based learning algorithm locality preserving graph-based semi-supervised method (LLGSM), which firstly use both labeled and unlabeled examples as unlabeled to discover the manifolds of the data samples and then use the projected labeled examples together with projected unlabeled ones to do classification. Experiments performed on some public image data sets have demonstrated the effectiveness of our algorithm.