Graph-adaptive Rectified Linear Unit for Graph Neural Networks

We investigate works under the propagation-based fake news detection domain, which recently seeks to improve performance through the use of Graph Neural Networks (GNNs). Generally, existing works argue that using GNNs can give results superior to what was obtained using classic graph-based methods. We agree with this argument given that GNNs are capable of gaining superior performance by leveraging node features. But we argue that existing works haven't identified the fact that the expressivity of GNNs is limited and bounded by node features. Existing works do not acknowledge that, by utilizing GNNs, they implicitly assume node features are strongly correlated to node labels. There are evidence that node features that have been employed do not necessarily correlate to node label. Instead of having a profound theoretical motivation, they have empirically observed that focusing on nodes features with strong feature-label correlation can increase predictive capability. This is a sub-optimal approach to view this problem, in fact, we argue that finding node features based on correlation is not practical or effective. Our first contribution is shifting readers from a node-level view i.e correlating node features with labels, to a graph-level view. In the graph-level view, we exploit the relationship between graph isomorphism and GNNs' expressivity which can be utilized to well understand and interpret the relation between node features and GNNs' expressivity. We conduct a wide range of experiments on basis of both node-level view and graph-level view and found graph-level view is more interpretable and strongly matches with results. Further, we gained insights on node features that wouldn't be obtainable by a node-level view. In order to have a fair and comprehensive analysis of node features, we built a unified dataset that includes a wide range of node features. Our results indicate, as we improve model accuracy on basis of the graph level view, models' generalizability decreases. We provide our hypothesis for this performance trade-off on the basis of the graph-level view. Our results and insights call for a much broader discussion on whether any sort of filtering method is effective. So, we conclude our work by providing readers with possible solutions that can be helpful to find harmony between node features and GNNs' expressivity.

Graph neural networks have been paid a lot attentions in recent years since many real-word data can naturally be represented by graph structures. Graph neural networks such as GCNs and GATs mainly focus on node features while ignoring edge features in graphs. However, in many graph structure data such as knowledge graphs, social networks, edge features are also important as they contain vital information about relations between nodes which are commonly ignored or simplified into binary or scalar values by existing methods. In this work we build a novel learning method on graphs called GNN-EE, i.e. GNN with Edge Enhanced, which takes both the node features and edge features into account while updating representations of graph components and can be applied to most of the common graph neural networks such as GCNs and GATs. Our GNN-EE method fits in the message-passing framework and thus is easy to generalize. In addition, we extend the random-walk-based algorithms on graphs so that they can consider both node and edge features on graphs. We use those random-walk-based algorithms as a pre-training method on graph with few initial features. We demonstrate the effectiveness and flexibility of our GNN-EE method through entity classification tasks and graph classification tasks.

Graph neural networks (GNNs) with missing node features have recently received increasing interest. Such missing node features seriously hurt the performance of the existing GNNs. Some recent methods have been proposed to reconstruct the missing node features by the information propagation among nodes with known and unknown attributes. Although these methods have achieved superior performance, how to exactly exploit the complex data correlations among nodes to reconstruct missing node features is still a great challenge. To solve the above problem, we propose a self-supervised guided hypergraph feature propagation (SGHFP). Specifically, the feature hypergraph is first generated according to the node features with missing information. And then, the reconstructed node features produced by the previous iteration are fed to a two-layer GNNs to construct a pseudo-label hypergraph. Before each iteration, the constructed feature hypergraph and pseudo-label hypergraph are fused effectively, which can better preserve the higher-order data correlations among nodes. After then, we apply the fused hypergraph to the feature propagation for reconstructing missing features. Finally, the reconstructed node features by multi-iteration optimization are applied to the downstream semi-supervised classification task. Extensive experiments demonstrate that the proposed SGHFP outperforms the existing semi-supervised classification with missing node feature methods.

Knowledge Graph Completion (KGC) is a popular topic in knowledge graph construction and related applications, aiming to complete the structure of knowledge graphs by predicting missing entities or relations and mining unknown facts in the knowledge graph. In the KGC task, Graph Neural Network (GNN)-based methods have achieved remarkable results due to their advantage of effectively capturing complex relations among entities and generating more accurate and rich entity representations by aggregating information from neighbouring nodes. These methods mainly focus on the representation of entities, and the representation of relations is obtained using simple dimensional transformations or initial embeddings. This treatment ignores the diversity and complex semantics of relations and restricts the efficiency of the model in utilizing relational information in the reasoning process. In this work, we propose the Relational Representation Augmented Graph Attention Network (RRA-GAT), which effectively identifies and weights neighbouring relations that actually contribute to the target relation by filtering out irrelevant information through an attention function based on the information and spatial domain. Furthermore, we capture complex patterns and features in the relational embedding by means of a feed-forward network consisting of a series of linear transformations and nonlinear activation functions. Experiments demonstrate the very advanced performance of RRA-GAT on the link prediction task on standard datasets FB15k-237 and WN18RR (e.g., improved the MRR metric on the WN18RR dataset by 7.8% relative improvement).

A Novel Privacy-Preserving Graph Convolutional Network via Secure Matrix Multiplication

Graph neural networks(GNNs) have been developed to solve challenging resource allocation (RA) problems, which leads to hopeful results in small and simple communication networks. Due to the inevitable heterogeneity of modern networks, it motivated researchers to develop the heterogeneous graph neural networks (HetGNNs) model for the RA problem of heterogeneous networks. However, node features and edge features are usually ignored by the most extant deep models, limited the performance when the size of the hidden layer in the network is larger than that of the node and edge features. In this paper, the power control or beamforming (PC/BF) in heterogeneous device-to-device (D2D) networks is focused, and proposed a HetGNN for the issue. Extensive simulations show that the proposed approach, matching or even outperforming state-of-the-art learning-based benchmarks.

In this paper, an improved Zhang neural network (IZNN) is proposed by using a kind of novel nonlinear activation function to solve the complex-valued systems of linear equation. Compared with the previous ZNN models, the convergence rate of the IZNN model has been accelerated. To do so, a kind of novel nonlinear activation function is first proposed to establish the novel recurrent neural network. Then, the corresponding maximum convergent time is given according to the randomly generated initial error vector, and the theoretical proof is described in detail in this paper. Finally, the experiment results illustrate that the new recurrent neural network using the proposed activation function has higher convergence rate than the previous neural networks using the linear activation function or the tunable activation function.

Graph Neural Networks (GNNs) have demonstrated significant success in various graph-related tasks. However, their performance is highly dependent on the availability of sufficient, class-balanced annotated data and an accurate graph structure. GNNs are vulnerable to substantial performance degradation when confronted with limited annotated samples, class imbalance, or noisy graph structures. First, a Dual GNN learning framework is introduced for semi-supervised node classification under limited supervision. The framework comprises two GNN-based node prediction modules: a primary module, which uses the input graph structure to induce standard node embeddings and predictions, and an auxiliary module, which leverages spectral clustering to construct a new graph structure and learn new node embeddings and predictions. These modules collaborate to enable end-to-end learning of discriminative node representations. Next, a graph augmentation method called Graph Dual Mixup (GDM) is proposed for graph classification with scarce labels. GDM employs a graph structural auto-encoder to learn structural embeddings, applying mixup in the learned structural embedding space to generate new graph structures. Additionally, mixup is applied to input node features, generating node features for new graph instances. Together, these generated node features and graph structures contribute new graphs that increase the size and diversity of the labeled dataset, improving classification performance. To enhance robustness against adversarial attacks on graph structures, an Efficient Low-Rank GNN is introduced. This approach learns robust low-rank sparse graph structures through a two-stage process. In the first stage, Singular Value Decomposition is used to estimate a low-rank approximation of the graph structure. This estimate is refined in the second stage by jointly learning a low-rank sparse graph structure alongside the GNN model, resulting in enhanced robustness. Finally, to address class-imbalanced node classification, a Unified GNN Learning (Uni-GNN) framework is proposed. Uni-GNN integrates structural and semantic connectivity representations to extend the propagation of embeddings to non-adjacent structural neighbors and semantically similar nodes, facilitating the diffusion of discriminative information. Additionally, it incorporates a balanced pseudo-label generation mechanism to improve the representations of minority classes. Comprehensive evaluations on benchmark datasets demonstrate that these methods outperform state-of-the-art approaches and hold significant potential for advancing GNN learning with limited supervision.

Entity Alignment (EA) aims to identify equivalent entities across different Knowledge Graphs (KGs), enabling knowledge fusion and integration. In recent years, Graph Neural Networks (GNNs) have emerged as a powerful paradigm for EA by leveraging structural information in KGs. However, most existing studies emphasize novel message passing mechanisms while overlooking other crucial GNN design components. This paper presents a comprehensive and systematic evaluation of GNN-based EA methods, focusing on three key aspects: message passing strategies, the number of GNN layers, and the construction of final entity representations. We benchmark a diverse set of GNN models originally developed for tasks such as node classification and knowledge graph completion, and we assess their adaptability to the EA task. Additionally, we explore the effectiveness of skip connection techniques, activation functions, and relational information integration. Our experiments, conducted on standard EA benchmarks including DBP15K and SRPRS, reveal several counterintuitive findings: (1) message passing is indispensable for EA; (2) many node classification GNNs are highly competitive for EA; (3) one or two GNN layers generally achieve optimal performance; and (4) activation functions have minimal impact, while skip connections significantly enhance results. This study provides a principled framework and empirical foundation for designing more effective GNN-based EA models. Code and datasets are publicly available at <uri xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://github.com/kg-bnu/GNN-EA">https://github.com/kg-bnu/GNN-EA</uri>.

Graph Neural Networks (GNNs) have emerged in recent years as a powerful tool to learn tasks across a wide range of graph domains in a data-driven fashion. Based on a message passing mechanism, GNNs have gained increasing popularity due to their intuitive formulation, closely linked to the Weisfeiler–Lehman (WL) test for graph isomorphism, to which they were demonstrated to be equivalent (Morris et al., 2019 and Xu et al., 2019). From a theoretical point of view, GNNs have been shown to be universal approximators, and their generalization capability — related to the Vapnik Chervonekis (VC) dimension (Scarselli et al., 2018) — has recently been investigated for GNNs with piecewise polynomial activation functions (Morris et al., 2023). The aim of our work is to extend this analysis on the VC dimension of GNNs to other commonly used activation functions, such as the sigmoid and hyperbolic tangent, using the framework of Pfaffian function theory. Bounds are provided with respect to the architecture parameters (depth, number of neurons, input size) as well as with respect to the number of colors resulting from the 1–WL test applied on the graph domain. The theoretical analysis is supported by a preliminary experimental study.

Real data collected from different applications that have additional topological structures and connection information are amenable to be represented as a weighted graph. Considering the node labeling problem, Graph Neural Networks (GNNs) is a powerful tool, which can mimic experts’ decision on node labeling. GNNs combine node features, connection patterns, and graph structure by using a neural network to embed node information and pass it through edges in the graph. We want to identify the patterns in the input data used by the GNN model to make a decision and examine if the model works as we desire. However, due to the complex data representation and non-linear transformations, explaining decisions made by GNNs is challenging. In this work, we propose new graph features’ explanation methods to identify the informative components and important node features. Besides, we propose a pipeline to identify the critical factors used for node classification. We use four datasets (two synthetic and two real) to validate our methods. Our results demonstrate that our explanation approach can mimic data patterns used for node classification by human interpretation and disentangle different features in the graphs. Furthermore, our explanation methods can be used for understanding data, debugging GNN models, and examine model decisions.

Cancer genomics is dedicated to elucidating the genes and pathways that contribute to cancer progression and development. Identifying cancer genes (CGs) associated with the initiation and progression of cancer is critical for characterization of molecular-level mechanism in cancer research. In recent years, the growing availability of high-throughput molecular data and advancements in deep learning technologies has enabled the modelling of complex interactions and topological information within genomic data. Nevertheless, because of the limited labelled data, pinpointing CGs from a multitude of potential mutations remains an exceptionally challenging task. To address this, we propose a novel deep learning framework, termed self-supervised masked graph learning (SMG), which comprises SMG reconstruction (pretext task) and task-specific fine-tuning (downstream task). In the pretext task, the nodes of multi-omic featured protein-protein interaction (PPI) networks are randomly substituted with a defined mask token. The PPI networks are then reconstructed using the graph neural network (GNN)-based autoencoder, which explores the node correlations in a self-prediction manner. In the downstream tasks, the pre-trained GNN encoder embeds the input networks into feature graphs, whereas a task-specific layer proceeds with the final prediction. To assess the performance of the proposed SMG method, benchmarking experiments are performed on three node-level tasks (identification of CGs, essential genes and healthy driver genes) and one graph-level task (identification of disease subnetwork) across eight PPI networks. Benchmarking experiments and performance comparison with existing state-of-the-art methods demonstrate the superiority of SMG on multi-omic feature engineering.

Signed attention based graph neural network for graphs with heterophily

Graphs help to define the relationships between entities in the data. These relationships, represented by edges, often provide additional context information which can be utilised to discover patterns in the data. Graph Neural Networks (GNNs) employ the inductive bias of the graph structure to learn and predict on various tasks. The primary operation of graph neural networks is the feature aggregation step performed over neighbours of the node based on the structure of the graph. In addition to its own features, for each hop, the node gets additional combined features from its neighbours. These aggregated features help define the similarity or dissimilarity of the nodes with respect to the labels and are useful for tasks like node classification. However, in real‐world data, features of neighbours at different hops may not correlate with the node's features. Thus, any indiscriminate feature aggregation by GNN might cause the addition of noisy features leading to degradation in model's performance. In this work, we show that selective aggregation of node features from various hops leads to better performance than default aggregation on the node classification task. Furthermore, we propose a Dual‐Net GNN architecture with a classifier model and a selector model. The classifier model trains over a subset of input node features to predict node labels while the selector model learns to provide optimal input subset to the classifier for the best performance. These two models are trained jointly to learn the best subset of features that give higher accuracy in node label predictions. With extensive experiments, we show that our proposed model outperforms both feature selection methods and state‐of‐the‐art GNN models with remarkable improvements up to 27.8%.

The graph neural network (GNN) has recently become an emerging research topic for processing non-euclidean data structures since the data used in various popular application domains are usually modeled as a graph, such as social networks, recommendation systems, and computer vision. Previous GNN accelerators commonly utilize the hybrid architecture to resolve the issue of “hybrid computing pattern” in GNN training. Nevertheless, the hybrid architecture suffers from poor utilization of hardware resources mainly due to the dynamic workloads between different phases in GNN. To address these issues, existing GNN accelerators adopt a unified structure with numerous processing elements and high bandwidth memory. However, the large amount of data movement between the processor and memory could heavily downgrade the performance of such accelerators in real-world graphs. As a result, the processing-in-memory architecture, such as the ReRAM-based crossbar, becomes a promising solution to reduce the memory overhead of GNN training. In this work, we present the TCAM-GNN, a novel TCAM-based data processing strategy, to enable high-throughput and energy-efficient GNN training over ReRAM-based crossbar architecture. Several hardware co-designed data structures and placement methods are proposed to fully exploit the parallelism in GNN during training. In addition, we propose a dynamic fixed-point formatting approach to resolve the precision issue. An adaptive data reusing policy is also proposed to enhance the data locality of graph features by the bootstrapping batch sampling approach. Overall, TCAM-GNN could enhance computing performance by 4.25× and energy efficiency by 9.11× on average compared to the neural network accelerators.