Sample-weighted fused graph-based semi-supervised learning on multi-view data

Semi-supervised learning with a graph-based approach has gained prominence in machine learning, particularly in scenarios where labeling data involves substantial costs. Graph convolution networks (GCNs) have found widespread application in semi-supervised learning, predominantly on graph-structured data such as citation and social networks. However, a noticeable gap exists in the application of these methods to non-graph multi-view data, such as collections of images. In an effort to address this gap, we introduce two innovative deep semi-supervised multi-view classification models specifically tailored for non-graph data. Both models share a common architecture, leveraging GCNs and integrating a label smoothing constraint. The primary distinction lies in the construction of the consensus similarity graph. The first model directly reconstructs the consensus graph from different views using a specialized objective function designed for flexible graph-based semi-supervised classification. In contrast, the second model independently reconstructs individual graphs and subsequently adaptively merges them into a unified consensus graph. Our experiments encompass various multiple-view image datasets. The results consistently demonstrate the superior performance of our proposed approach compared to traditional fusion methods with GCNs. In this research, we present two approaches for tackling semi-supervised classification challenges involving multiple views. One method is named Semi-supervised Classification with a Unified Graph (SCUG), and the other is referred to as Semi-supervised Classification with a Fused Graph (SC-Fused). Both methods share a common semi-supervised classification process, utilizing the GCN framework and incorporating label smoothing. However, the key distinction lies in the construction of the similarity graph. Unlike traditional ad hoc graph construction approaches, our proposed methods, SCUG and SC-Fused, estimate the unified graph or individual graphs, respectively, alongside the labels. This results in more optimized graphs that benefit from data smoothing and the semi-supervised context.

Joint learning of feature and topology for multi-view graph convolutional network

A new graph-based semi-supervised method for surface defect classification

ABSTRACTIn the area of hyperspectral image (HSI) classification, graph-based semi-supervised learning (SSL) has been proved to be highly effective. Constructing a proper graph is critical for graph-based SSL tasks. In HSI, spectral distance is widely used to calculate the weight of graph edge, though it can be influenced by noise and outliers. Meanwhile, links among all the data points are incorporated in the graph, including those from different subspaces. Thus the constructed graph might contain incorrect information. In this letter, a novel semi-supervised HSI classification method using local low-rank representation (SL2R) is proposed. Edge weight calculation will not be affected by noise or outliers thanks to the robustness of low-rank representation (LRR). Since each graph is constructed at local level, where pixels are basically embedded in the same subspace, links among uncorrelated pixels can be removed. Moreover, spatial context is naturally characterized by low-rank constraint on adjacent pixels. Experimental results on two data sets (Indian Pines and Botswana) confirm the effectiveness of the proposed method.

Semi-supervised classification on data streams with recurring concept drift and concept evolution

New semi-supervised classification using a multi-modal feature joint [formula omitted]-norm based sparse representation

In recent years, substantial progress has been made on Graph Convolutional Networks (GCNs). However, the computing of GCN usually requires a large memory space for keeping the entire graph. In consequence, GCN is not flexible enough, especially for large scale graphs in complex real-world applications. Fortunately, for transductive graph representation learning, methods based on Matrix Factorization (MF) naturally support constructing mini-batches, and thus are more friendly to distributed computing compared with GCN. Accordingly, in this paper, we analyze the connections between GCN and MF, and simplify GCN as matrix factorization with unitization and co-training. Furthermore, under the guidance of our analysis, we propose an alternative model to GCN named Unitized and Co-training Matrix Factorization (UCMF). Extensive experiments have been conducted on several real-world datasets. On the task of semi-supervised node classification, the experimental results illustrate that UCMF achieves similar or superior performances compared with GCN. Meanwhile, distributed UCMF significantly outperforms distributed GCN methods, which shows that UCMF can greatly benefit complex real-world applications.

Real-world applications have produced massive short text streams. Contrary to the traditional normal texts, they present the characteristics such as short length, only having few labeled data, high-velocity, high-volume and dynamic data distributions, which deteriorate the issues of data sparseness, label missing and concept drift. Obviously, it is a huge challenge for existing short text (stream) classification algorithms due to the poor effectiveness, where they always assume all short texts are completely labeled and little attention is paid on the concept drift issue hidden in short text streams. Therefore, we propose a novel semi-supervised short text steam classification method based on the drift-aware incremental deep learning ensemble model. Specifically, with the sliding window mechanism, we firstly fuse three types of statistical, semantic and structure information to solve the data sparseness issue. Secondly, a semi-supervised incremental deep learning ensemble model based on GCN and the refined LSTM is developed to adapt to the high-volume, high-velocity and label missing short text streams. Thirdly, a label-probability distribution based concept drift detector is introduced to distinguish concept drifts. Finally, as compared with eleven well-known classification methods, extensive experiments demonstrate the effectiveness of the proposed method in the handling of short text streams with limited labeled data.

Graph Convolutional Network (GCN) is a widely used method for learning from graph-based data. However, it fails to use the unlabeled data to its full potential, thereby hindering its ability. Given some pseudo labels of the unlabeled data, the GCN can benefit from this extra supervision. Based on Knowledge Distillation and Ensemble Learning, lots of methods use a teacher-student architecture to make better use of the unlabeled data and then make a better prediction. However, these methods introduce unnecessary training costs and a high bias of student model if the teacher's predictions are unreliable. Besides, the final ensemble gains are limited due to limited diversity in the combined models. Therefore, we propose Reliable Data Distillation, a reliable data driven semi-supervised GCN training method. By defining the node reliability and edge reliability in a graph, we can make better use of high quality data and improve the graph representation learning. Furthermore, considering the data reliability and data importance, we propose a new ensemble learning method for GCN and a novel Self-Boosting SSL Framework to combine the above optimizations. Finally, our extensive evaluation of Reliable Data Distillation on real-world datasets shows that our approach outperforms the state-of-the-art methods on semi-supervised node classification tasks.

Over the past few years, graph representation learning (GRL) has received widespread attention on the feature representations of the non-Euclidean data. As a typical model of GRL, graph convolutional networks (GCN) fuse the graph Laplacian-based static sample structural information. GCN thus generalizes convolutional neural networks to acquire the sample representations with the variously high-order structures. However, most of existing GCN-based variants depend on the static data structural relationships. It will result in the extracted data features lacking of representativeness during the convolution process. To solve this problem, dynamic graph learning convolutional networks (DGLCN) on the application of semi-supervised classification are proposed. First, we introduce a definition of dynamic spectral graph convolution operation. It constantly optimizes the high-order structural relationships between data points according to the loss values of the loss function, and then fits the local geometry information of data exactly. After optimizing our proposed definition with the one-order Chebyshev polynomial, we can obtain a single-layer convolution rule of DGLCN. Due to the fusion of the optimized structural information in the learning process, multi-layer DGLCN can extract richer sample features to improve classification performance. Substantial experiments are conducted on citation network datasets to prove the effectiveness of DGLCN. Experiment results demonstrate that the proposed DGLCN obtains a superior classification performance compared to several existing semi-supervised classification models.

Short text classification has found rich and critical applications in news and tweet tagging to help users find relevant information. Due to lack of labeled training data in many practical use cases, there is a pressing need for studying semi-supervised short text classification. Most existing studies focus on long texts and achieve unsatisfactory performance on short texts due to the sparsity and limited labeled data. In this paper, we propose a novel heterogeneous graph neural network based method for semi-supervised short text classification, leveraging full advantage of few labeled data and large unlabeled data through information propagation along the graph. In particular, we first present a flexible HIN (heterogeneous information network) framework for modeling the short texts, which can integrate any type of additional information as well as capture their relations to address the semantic sparsity. Then, we propose Heterogeneous Graph ATtention networks (HGAT) to embed the HIN for short text classification based on a dual-level attention mechanism, including node-level and type-level attentions. The attention mechanism can learn the importance of different neighboring nodes as well as the importance of different node (information) types to a current node. Extensive experimental results have demonstrated that our proposed model outperforms state-of-the-art methods across six benchmark datasets significantly.

A scalable semisupervised node classification method on graph-structured data, called GraphHop, is proposed in this work. The graph contains all nodes' attributes and link connections but labels of only a subset of nodes. Graph convolutional networks (GCNs) have provided superior performance in node label classification over the traditional label propagation (LP) methods for this problem. Nevertheless, current GCN algorithms suffer from a considerable amount of labels for training because of high model complexity or cannot be easily generalized to large-scale graphs due to the expensive cost of loading the entire graph and node embeddings. Besides, nonlinearity makes the optimization process a mystery. To this end, an enhanced LP method, called GraphHop, is proposed to tackle these problems. GraphHop can be viewed as a smoothening LP algorithm, in which each propagation alternates between two steps: label aggregation and label update. In the label aggregation step, multihop neighbor embeddings are aggregated to the center node. In the label update step, new embeddings are learned and predicted for each node based on aggregated results from the previous step. The two-step iteration improves the graph signal smoothening capacity. Furthermore, to encode attributes, links, and labels on graphs effectively under one framework, we adopt a two-stage training process, i.e., the initialization stage and the iteration stage. Thus, the smooth attribute information extracted from the initialization stage is consistently imposed in the propagation process in the iteration stage. Experimental results show that GraphHop outperforms state-of-the-art graph learning methods on a wide range of tasks in graphs of various sizes (e.g., multilabel and multiclass classification on citation networks, social graphs, and commodity consumption graphs).

As a current research hotspot, graph convolution networks (GCNs) have provided new opportunities for tree species classification in multi-source remote sensing images. To solve the challenge of limited label information, a new tree species classification model was proposed by using the semi-supervised graph convolution fusion method for hyperspectral images (HSIs) and multispectral images (MSIs). In the model, the graph-based attribute features and pixel-based features are fused to deepen the correlation of multi-source images to improve accuracy. Firstly, the model employs the canonical correlation analysis (CCA) method to maximize the correlation of multi-source images, which explores the relationship between information from various sources further and offers more valuable insights. Secondly, convolution calculations were made to extract features and then map graph node fusion, which not only reduces redundancy features but also enhances compelling features. Finally, the relationship between representative descriptors is captured through the use of hyperedge convolution in the training process, and the dominant features on the graph are fully mined. The tree species are classified through two fusion feature operations, leading to improved classification performance compared to state-of-the-art methods. The fusion strategy can produce a complete classification map of the study areas.

Graph-based semi-supervised learning (SSL) algorithms have been widely studied in the last few years. Most of these algorithms were designed from unconstrained optimization problems using a Laplacian regularizer term as smoothness functional in an attempt to reflect the intrinsic geometric structure of the data's marginal distribution. Although a number of recent research papers are still focusing on unconstrained methods for graph-based SSL, a recent statistical analysis showed that many of these algorithms may be unstable on transductive regression. Therefore, we focus on providing new constrained methods for graph-based SSL. We begin by analyzing the regularization framework of existing unconstrained methods. Then, we incorporate two normalization constraints into the optimization problem of three of these methods. We show that the proposed optimization problems have closed-form solution. By generalizing one of these constraints to any distribution, we provide generalized methods for constrained graph-based SSL. The proposed methods have a more flexible regularization framework than the corresponding unconstrained methods. More precisely, our methods can deal with any graph Laplacian and use higher order regularization, which is effective on general SSL taks. In order to show the effectiveness of the proposed methods, we provide comprehensive experimental analyses. Specifically, our experiments are subdivided into two parts. In the first part, we evaluate existing graph-based SSL algorithms on time series data to find their weaknesses. In the second part, we evaluate the proposed constrained methods against six state-of-the-art graph-based SSL algorithms on benchmark data sets. Since the widely used best case analysis may hide useful information concerning the SSL algorithms' performance with respect to parameter selection, we used recently proposed empirical evaluation models to evaluate our results. Our results show that our methods outperforms the competing methods on most parameter settings and graph construction methods. However, we found a few experimental settings in which our methods showed poor performance. In order to facilitate the reproduction of our results, the source codes, data sets, and experimental results are freely available.

Semi-supervised learning is drawing increasing attention in the era of big data, as the gap between the abundance of cheap, automatically collected unlabeled data and the scarcity of labeled data that are laborious and expensive to obtain is dramatically increasing. In this thesis, we first introduce a unified view of density-based clustering algorithms. Then, we build upon this view and bridge the areas of semi-supervised clustering and classification under a common umbrella of density-based techniques. We show that there are close relations between density-based clustering algorithms and the graph-based approach for transductive classification. These relations are then used as a basis for a new framework for semi-supervised classification based on building-blocks from density-based clustering. This framework is not only efficient and effective, but it is also statistically sound. We also generalize the core algorithm of the framework HDBSCAN* so that it can also perform semi-supervised clustering by directly taking advantage of any fraction of labeled data that may be available, rather than instance-level pairwise constraints. Experimental results on a large collection of datasets show the advantages of the proposed approach both for semi-supervised classification, as well as for semi-supervised clustering. In addition, we evaluate the semi-supervised learning algorithms to determine relationships between chemical structure and biological activity in datasets from Medicinal Chemistry. The datasets evaluated in this area are characterized by a low number of labeled examples, a high dimensionality, and in some cases, do not have a clear relationship between chemical structure and biological activity, which makes it difficult to use classification techniques and analyze chemical phenomena. We implement and validate semi-supervised classification approaches that are appropriate for data analysis in Medicinal Chemistry.