- Research Article
8
- 10.1016/j.neucom.2021.08.092
Entropy-aware self-training for graph convolutional networks
- Aug 25, 2021
- Neurocomputing
- Gongpei Zhao + 4 more +4
Entropy-aware self-training for graph convolutional networks
Graph convolutional networks (GCNs) are widely believed to perform well in the graph node classification task, and homophily assumption plays a core rule in the design of previous GCNs. However, some recent advances on this area have pointed out that homophily may not be a necessity for GCNs. For deeper analysis of the critical factor affecting the performance of GCNs, we first propose a metric, namely, neighborhood class consistency (NCC), to quantitatively characterize the neighborhood patterns of graph datasets. Experiments surprisingly illustrate that our NCC is a better indicator, in comparison to the widely used homophily metrics, to estimate GCN performance for node classification. Furthermore, we propose a topology augmentation graph convolutional network (TA-GCN) framework under the guidance of the NCC metric, which simultaneously learns an augmented graph topology with higher NCC score and a node classifier based on the augmented graph topology. Extensive experiments on six public benchmarks clearly show that the proposed TA-GCN derives ideal topology with higher NCC score given the original graph topology and raw features, and it achieves excellent performance for semi-supervised node classification in comparison to several state-of-the-art (SOTA) baseline algorithms.
Entropy-aware self-training for graph convolutional networks
Entropy-aware self-training for graph convolutional networks
Simplifying Graph Convolutional Networks as Matrix Factorization
In recent years, substantial progress has been made on Graph Convolutional Networks (GCNs). However, the computing of GCN usually requires a large memory space for keeping the entire graph. In consequence, GCN is not flexible enough, especially for large scale graphs in complex real-world applications. Fortunately, for transductive graph representation learning, methods based on Matrix Factorization (MF) naturally support constructing mini-batches, and thus are more friendly to distributed computing compared with GCN. Accordingly, in this paper, we analyze the connections between GCN and MF, and simplify GCN as matrix factorization with unitization and co-training. Furthermore, under the guidance of our analysis, we propose an alternative model to GCN named Unitized and Co-training Matrix Factorization (UCMF). Extensive experiments have been conducted on several real-world datasets. On the task of semi-supervised node classification, the experimental results illustrate that UCMF achieves similar or superior performances compared with GCN. Meanwhile, distributed UCMF significantly outperforms distributed GCN methods, which shows that UCMF can greatly benefit complex real-world applications.
Read moreGCN-LRP explanation: exploring latent attention of graph convolutional networks
Graph convolutional networks (GCNs) have been successfully applied to many graph data on various learning tasks such as node classification. However, there is limited understanding of the internal logic and decision patterns of GCNs. In this paper, we propose a layer-wise relevance propagation based explanation method for GCNs, namely GCN-LRP, to explore the latent pattern of GCNs. Then, we use three well-known citation network data sets and synthetic graph data sets for node classification tasks with GCN-LRP explanation, and experimentally identify latent attentions when GCNs aggregates information from the node and its neighboring nodes: (i) GCNs pay more attention to the classified node when comparing with its neighbors; (ii) GCNs do not pay attention to all the neighboring nodes equally, and a few neighboring nodes received more attention than others. Moreover, we further theoretically analyze and find that: (i) the latent attentions come from the recursively aggregating of GCNs; (ii) the neighboring nodes, which share enough neighbors with classified node, would receive more attention than other neighbors; (iii) the latent attention could hardly be changed by model training. We also discuss the advantage and limitations of GCNs introduced by the latent attentions, and implications of our findings for graph data learning with GCNs.
Read moreLeveraging Graph Convolutional Networks for Semi-supervised Learning in Multi-view Non-graph Data
Semi-supervised learning with a graph-based approach has gained prominence in machine learning, particularly in scenarios where labeling data involves substantial costs. Graph convolution networks (GCNs) have found widespread application in semi-supervised learning, predominantly on graph-structured data such as citation and social networks. However, a noticeable gap exists in the application of these methods to non-graph multi-view data, such as collections of images. In an effort to address this gap, we introduce two innovative deep semi-supervised multi-view classification models specifically tailored for non-graph data. Both models share a common architecture, leveraging GCNs and integrating a label smoothing constraint. The primary distinction lies in the construction of the consensus similarity graph. The first model directly reconstructs the consensus graph from different views using a specialized objective function designed for flexible graph-based semi-supervised classification. In contrast, the second model independently reconstructs individual graphs and subsequently adaptively merges them into a unified consensus graph. Our experiments encompass various multiple-view image datasets. The results consistently demonstrate the superior performance of our proposed approach compared to traditional fusion methods with GCNs. In this research, we present two approaches for tackling semi-supervised classification challenges involving multiple views. One method is named Semi-supervised Classification with a Unified Graph (SCUG), and the other is referred to as Semi-supervised Classification with a Fused Graph (SC-Fused). Both methods share a common semi-supervised classification process, utilizing the GCN framework and incorporating label smoothing. However, the key distinction lies in the construction of the similarity graph. Unlike traditional ad hoc graph construction approaches, our proposed methods, SCUG and SC-Fused, estimate the unified graph or individual graphs, respectively, alongside the labels. This results in more optimized graphs that benefit from data smoothing and the semi-supervised context.
Read moreSelf-attention Based Multi-scale Graph Convolutional Networks
Graph convolutional networks (GCNs) have achieved remarkable learning ability for dealing with various graph structural data recently. In general, GCNs have low expressive power due to their shallow structure. In this paper, to improve the expressive power of GCNs, we propose two multi-scale GCN frameworks by incorporating self-attention mechanism and multi-scale information into the design of GCNs. The self-attention mechanism allows us to adaptively learn the local structure of the neighborhood, and achieves more accurate predictions. Extensive experiments on both node classification and graph classification demonstrate the effectiveness of our approaches over several state-of-the-art GCNs.
Read moreGraphHop: An Enhanced Label Propagation Method for Node Classification.
A scalable semisupervised node classification method on graph-structured data, called GraphHop, is proposed in this work. The graph contains all nodes' attributes and link connections but labels of only a subset of nodes. Graph convolutional networks (GCNs) have provided superior performance in node label classification over the traditional label propagation (LP) methods for this problem. Nevertheless, current GCN algorithms suffer from a considerable amount of labels for training because of high model complexity or cannot be easily generalized to large-scale graphs due to the expensive cost of loading the entire graph and node embeddings. Besides, nonlinearity makes the optimization process a mystery. To this end, an enhanced LP method, called GraphHop, is proposed to tackle these problems. GraphHop can be viewed as a smoothening LP algorithm, in which each propagation alternates between two steps: label aggregation and label update. In the label aggregation step, multihop neighbor embeddings are aggregated to the center node. In the label update step, new embeddings are learned and predicted for each node based on aggregated results from the previous step. The two-step iteration improves the graph signal smoothening capacity. Furthermore, to encode attributes, links, and labels on graphs effectively under one framework, we adopt a two-stage training process, i.e., the initialization stage and the iteration stage. Thus, the smooth attribute information extracted from the initialization stage is consistently imposed in the propagation process in the iteration stage. Experimental results show that GraphHop outperforms state-of-the-art graph learning methods on a wide range of tasks in graphs of various sizes (e.g., multilabel and multiclass classification on citation networks, social graphs, and commodity consumption graphs).
Read moreCross-Space Adaptive Filter: Integrating Graph Topology and Node Attributes for Alleviating the Over-smoothing Problem
The vanilla Graph Convolutional Network (GCN) uses a low-pass filter to extract low-frequency signals from graph topology, which may lead to the over-smoothing problem when GCN goes deep. To this end, various methods have been proposed to create an adaptive filter by incorporating an extra filter (e.g., a high-pass filter) extracted from the graph topology. However, these methods heavily rely on topological information and ignore the node attribute space, which severely sacrifices the expressive power of the deep GCNs, especially when dealing with disassortative graphs. In this paper, we propose a cross-space adaptive filter, called CSF, to produce the adaptive-frequency information extracted from both the topology and attribute spaces. Specifically, we first derive a tailored attribute-based high-pass filter that can be interpreted theoretically as a minimizer for semi-supervised kernel ridge regression. Then, we cast the topology-based low-pass filter as a Mercer's kernel within the context of GCNs. This serves as a foundation for combining it with the attribute-based filter to capture the adaptive-frequency information. Finally, we derive the cross-space filter via an effective multiple-kernel learning strategy, which unifies the attribute-based high-pass filter and the topology-based low-pass filter. This helps to address the over-smoothing problem while maintaining effectiveness. Extensive experiments demonstrate that CSF not only successfully alleviates the over-smoothing problem but also promotes the effectiveness of the node classification task. Our code is available at https://github.com/huangzichun/Cross-Space-Adaptive-Filter.
Read moreSemi-Supervised Tree Species Classification for Multi-Source Remote Sensing Images Based on a Graph Convolutional Neural Network
As a current research hotspot, graph convolution networks (GCNs) have provided new opportunities for tree species classification in multi-source remote sensing images. To solve the challenge of limited label information, a new tree species classification model was proposed by using the semi-supervised graph convolution fusion method for hyperspectral images (HSIs) and multispectral images (MSIs). In the model, the graph-based attribute features and pixel-based features are fused to deepen the correlation of multi-source images to improve accuracy. Firstly, the model employs the canonical correlation analysis (CCA) method to maximize the correlation of multi-source images, which explores the relationship between information from various sources further and offers more valuable insights. Secondly, convolution calculations were made to extract features and then map graph node fusion, which not only reduces redundancy features but also enhances compelling features. Finally, the relationship between representative descriptors is captured through the use of hyperedge convolution in the training process, and the dominant features on the graph are fully mined. The tree species are classified through two fusion feature operations, leading to improved classification performance compared to state-of-the-art methods. The fusion strategy can produce a complete classification map of the study areas.
Read moreDynamic Graph Learning Convolutional Networks for Semi-supervised Classification
Over the past few years, graph representation learning (GRL) has received widespread attention on the feature representations of the non-Euclidean data. As a typical model of GRL, graph convolutional networks (GCN) fuse the graph Laplacian-based static sample structural information. GCN thus generalizes convolutional neural networks to acquire the sample representations with the variously high-order structures. However, most of existing GCN-based variants depend on the static data structural relationships. It will result in the extracted data features lacking of representativeness during the convolution process. To solve this problem, dynamic graph learning convolutional networks (DGLCN) on the application of semi-supervised classification are proposed. First, we introduce a definition of dynamic spectral graph convolution operation. It constantly optimizes the high-order structural relationships between data points according to the loss values of the loss function, and then fits the local geometry information of data exactly. After optimizing our proposed definition with the one-order Chebyshev polynomial, we can obtain a single-layer convolution rule of DGLCN. Due to the fusion of the optimized structural information in the learning process, multi-layer DGLCN can extract richer sample features to improve classification performance. Substantial experiments are conducted on citation network datasets to prove the effectiveness of DGLCN. Experiment results demonstrate that the proposed DGLCN obtains a superior classification performance compared to several existing semi-supervised classification models.
Read moreIMG-80. Non-Invasive Classification of Glioblastoma Subtypes at the Infiltrative Tumor Margin Using Multiparametric MRI and Graph Convolutional Networks
BACKGROUND Glioblastoma (GBM) is a molecularly and phenotypically heterogeneous tumor, with regional invasion patterns driving treatment resistance and universal recurrence. In infiltrative regions, GBM cells adopt at least two predominant invasive phenotypes: Neuronal-enriched (NEU), with synaptic and neurodevelopment programs, and Glycolytic/Plurimetabolic (GPM), with heightened immune and metabolic activity. These subtypes differ in therapeutic vulnerability and microenvironmental interactions; features not captured by conventional contrast-enhanced (T1+C) MRI due to minimal blood-brain barrier (BBB) disruption. Our group sought to develop predictive models to distinguish variability of NEU/GPM subtypes within infiltrative GBM, utilizing graph convolutional networks (GCN), advanced multiparametric MRI (mpMRI), and spatially-matched image-localized biopsies with transcriptomic profiling. METHODS We analyzed 18 mpMRI features —including conventional, diffusion-weighted, and Dynamic Susceptibility Contrast (DSC) perfusion MRI— of 101 image-localized biopsies from MRI non-enhancing infiltrative margins of 64 GBM patient tumors. Features were hierarchically clustered per modality; each cluster was represented as a node in a Neo4j graph. Edges connected nodes sharing overlapping samples. Nodes were annotated with NEU or GPM subtype proportion. GCNs were trained to classify subtype-enriched nodes using graph topology and node features. RESULTS GCNs achieved high predictive performance on held-out test data (AUC = 0.95 for NEU, 0.88 for GPM; 70:30 train-test split) with similar 5-fold cross-validation performance and generalized well to an independent cohort (AUC > 0.85, n = 45). Importantly, performance was robust without relying on T1+C imaging, enabling prediction beyond BBB-disrupted regions. GCNs outperformed traditional classifiers (logistic regression, random forest). CONCLUSION Our results demonstrate that GCN-modeling of mpMRI features enables non-invasive prediction of invasive GBM subtypes at the tumor margin. Unlike convolutional neural networks (CNN), which rely on spatial continuity, GCNs leverage relational structure between heterogeneous imaging features, making them ideal for modeling sparse, non-grid data. This may support non-invasive regional vulnerability detection, surgical targeting, and improved response assessment.
Read moreLightweight Connectivity In Graph Convolutional Networks For Skeleton-Based Recognition
Graph convolutional networks (GCNs) aim at extending deep learning to arbitrary irregular domains, namely graphs. Their success is highly dependent on how the topology of input graphs is defined and most of the existing GCN architectures rely on predefined or handcrafted graph structures. In this paper, we introduce a novel method that learns the topology (or connectivity) of input graphs as a part of GCN design. The main contribution of our method resides in building an orthogonal connectivity basis that optimally aggregates nodes, through their neighborhood, prior to achieve convolution. Our method also considers a stochasticity criterion which acts as a regularizer that makes the learned basis and the underlying GCNs lightweight while still being highly effective. Experiments conducted on the challenging task of skeleton-based hand-gesture recognition show the high effectiveness of the learned GCNs w.r.t. the related work.
Read moreHDGCN: Dual-Channel Graph Convolutional Network With Higher-Order Information for Robust Feature Learning
Graph convolutional network (GCN) algorithms have been employed to learn graph embedding due to its inductive inference property, which is extended to GCN with higher-order information. However, the application of these higher-order information is confusing and not effectively distinguished. In addition, single channel GCN using higher-order information is weak for robust feature learning, and existing dual-channel GCNs rarely take higher-order information into account. To alleviate the above problems, we propose a dual-channel GCN with higher-order information for robust feature learning, denoted as HDGCN. Firstly, features of positive and negative higher-order graphs are extracted that fully exploits the self-contained attributes and higher-order geometric information. Meantime, the features of original graph structure are extracted by a conventional GCN that utilizes the self-contained feature attributes. Then, node features are represented as edge features by a feature fusion function. For the selection of negative samples, a fractional staggered negative sampling method is applied, by which the trainable graph model gains better topological features. Finally, the performance on seven real-world datasets demonstrates that HDGCN obtains the state-of-the-art performance on pairwise link prediction, higher-order structure prediction, and node classification tasks. By changing the attributes of multiple tasks, it can be proved that HDGCN has good robustness.
Read moreMDA-GCNFTG: identifying miRNA-disease associations based on graph convolutional networks via graph sampling through the feature and topology graph.
Accurate identification of the miRNA-disease associations (MDAs) helps to understand the etiology and mechanisms of various diseases. However, the experimental methods are costly and time-consuming. Thus, it is urgent to develop computational methods towards the prediction of MDAs. Based on the graph theory, the MDA prediction is regarded as a node classification task in the present study. To solve this task, we propose a novel method MDA-GCNFTG, which predicts MDAs based on Graph Convolutional Networks (GCNs) via graph sampling through the Feature and Topology Graph to improve the training efficiency and accuracy. This method models both the potential connections of feature space and the structural relationships of MDA data. The nodes of the graphs are represented by the disease semantic similarity, miRNA functional similarity and Gaussian interaction profile kernel similarity. Moreover, we considered six tasks simultaneously on the MDA prediction problem at the first time, which ensure that under both balanced and unbalanced sample distribution, MDA-GCNFTG can predict not only new MDAs but also new diseases without known related miRNAs and new miRNAs without known related diseases. The results of 5-fold cross-validation show that the MDA-GCNFTG method has achieved satisfactory performance on all six tasks and is significantly superior to the classic machine learning methods and the state-of-the-art MDA prediction methods. Moreover, the effectiveness of GCNs via the graph sampling strategy and the feature and topology graph in MDA-GCNFTG has also been demonstrated. More importantly, case studies for two diseases and three miRNAs are conducted and achieved satisfactory performance.
Read moreDistributed Training of Graph Convolutional Networks
The aim of this work is to develop a fully-distributed algorithmic framework for training graph convolutional networks (GCNs). The proposed method is able to exploit the meaningful relational structure of the input data, which are collected by a set of agents that communicate over a sparse network topology. After formulating the centralized GCN training problem, we first show how to make inference in a distributed scenario where the underlying data graph is split among different agents. Then, we propose a distributed gradient descent procedure to solve the GCN training problem. The resulting model distributes computation along three lines: during inference, during back-propagation, and during optimization. Convergence to stationary solutions of the GCN training problem is also established under mild conditions. Finally, we propose an optimization criterion to design the communication topology between agents in order to match with the graph describing data relationships. A wide set of numerical results validate our proposal. To the best of our knowledge, this is the first work combining graph convolutional neural networks with distributed optimization.
Read moreDetection of Discriminative Neurological Circuits Using Hierarchical Graph Convolutional Networks in fMRI Sequences
Graph convolutional network (GCN) has shown its potential on modeling functional MRI connectivity and recognizing neurological disease tasks. However, conventional GCN layers generally inherit the original graph topology, without the modeling of hierarchical graph representation. Besides, although the interpretability of GCN has been widely investigated, such studies only identify several independently affected brain regions instead of forming them as neurological circuits, which are more desirable for disease mechanism investigation. In this paper, we propose a hierarchical dynamic GCN (HD-GCN), which combines the information from both low-order graph composed of brain regions and high-order graph composed of brain region clusters. The algorithm learns a consistent dynamic graph pooling, which helps improve the classification accuracy by hierarchical graph representation learning and could identify the affected neurological circuits. We employed two datasets to evaluate the generalizability of the proposed method: ADNI dataset containing 177 AD patients and 115 controls, and Obsessive-Compulsive Disorder (OCD) dataset including 67 patients and 61 controls. The classification accuracy reaches \(89.4\%\) on ADNI dataset and \(89.1\%\) on OCD dataset. The affected brain circuits were also identified, which are consistent with previous psychological studies.
Read more