A Piece-Wise Polynomial Filtering Approach for Graph Neural Networks

We investigate works under the propagation-based fake news detection domain, which recently seeks to improve performance through the use of Graph Neural Networks (GNNs). Generally, existing works argue that using GNNs can give results superior to what was obtained using classic graph-based methods. We agree with this argument given that GNNs are capable of gaining superior performance by leveraging node features. But we argue that existing works haven't identified the fact that the expressivity of GNNs is limited and bounded by node features. Existing works do not acknowledge that, by utilizing GNNs, they implicitly assume node features are strongly correlated to node labels. There are evidence that node features that have been employed do not necessarily correlate to node label. Instead of having a profound theoretical motivation, they have empirically observed that focusing on nodes features with strong feature-label correlation can increase predictive capability. This is a sub-optimal approach to view this problem, in fact, we argue that finding node features based on correlation is not practical or effective. Our first contribution is shifting readers from a node-level view i.e correlating node features with labels, to a graph-level view. In the graph-level view, we exploit the relationship between graph isomorphism and GNNs' expressivity which can be utilized to well understand and interpret the relation between node features and GNNs' expressivity. We conduct a wide range of experiments on basis of both node-level view and graph-level view and found graph-level view is more interpretable and strongly matches with results. Further, we gained insights on node features that wouldn't be obtainable by a node-level view. In order to have a fair and comprehensive analysis of node features, we built a unified dataset that includes a wide range of node features. Our results indicate, as we improve model accuracy on basis of the graph level view, models' generalizability decreases. We provide our hypothesis for this performance trade-off on the basis of the graph-level view. Our results and insights call for a much broader discussion on whether any sort of filtering method is effective. So, we conclude our work by providing readers with possible solutions that can be helpful to find harmony between node features and GNNs' expressivity.

Graph neural networks have been paid a lot attentions in recent years since many real-word data can naturally be represented by graph structures. Graph neural networks such as GCNs and GATs mainly focus on node features while ignoring edge features in graphs. However, in many graph structure data such as knowledge graphs, social networks, edge features are also important as they contain vital information about relations between nodes which are commonly ignored or simplified into binary or scalar values by existing methods. In this work we build a novel learning method on graphs called GNN-EE, i.e. GNN with Edge Enhanced, which takes both the node features and edge features into account while updating representations of graph components and can be applied to most of the common graph neural networks such as GCNs and GATs. Our GNN-EE method fits in the message-passing framework and thus is easy to generalize. In addition, we extend the random-walk-based algorithms on graphs so that they can consider both node and edge features on graphs. We use those random-walk-based algorithms as a pre-training method on graph with few initial features. We demonstrate the effectiveness and flexibility of our GNN-EE method through entity classification tasks and graph classification tasks.

Graph neural networks induced by concept lattices for classification

Graph neural networks(GNNs) have been developed to solve challenging resource allocation (RA) problems, which leads to hopeful results in small and simple communication networks. Due to the inevitable heterogeneity of modern networks, it motivated researchers to develop the heterogeneous graph neural networks (HetGNNs) model for the RA problem of heterogeneous networks. However, node features and edge features are usually ignored by the most extant deep models, limited the performance when the size of the hidden layer in the network is larger than that of the node and edge features. In this paper, the power control or beamforming (PC/BF) in heterogeneous device-to-device (D2D) networks is focused, and proposed a HetGNN for the issue. Extensive simulations show that the proposed approach, matching or even outperforming state-of-the-art learning-based benchmarks.

Graphs help to define the relationships between entities in the data. These relationships, represented by edges, often provide additional context information which can be utilised to discover patterns in the data. Graph Neural Networks (GNNs) employ the inductive bias of the graph structure to learn and predict on various tasks. The primary operation of graph neural networks is the feature aggregation step performed over neighbours of the node based on the structure of the graph. In addition to its own features, for each hop, the node gets additional combined features from its neighbours. These aggregated features help define the similarity or dissimilarity of the nodes with respect to the labels and are useful for tasks like node classification. However, in real‐world data, features of neighbours at different hops may not correlate with the node's features. Thus, any indiscriminate feature aggregation by GNN might cause the addition of noisy features leading to degradation in model's performance. In this work, we show that selective aggregation of node features from various hops leads to better performance than default aggregation on the node classification task. Furthermore, we propose a Dual‐Net GNN architecture with a classifier model and a selector model. The classifier model trains over a subset of input node features to predict node labels while the selector model learns to provide optimal input subset to the classifier for the best performance. These two models are trained jointly to learn the best subset of features that give higher accuracy in node label predictions. With extensive experiments, we show that our proposed model outperforms both feature selection methods and state‐of‐the‐art GNN models with remarkable improvements up to 27.8%.

Graph Neural Networks (GNNs) have achieved remarkable success in recent years due to their powerful ability to model non-Euclidean data structures and complex relationships. However, as graph sizes and model complexities continue to grow, the efficient deployment of GNNs across various hardware platforms presents significant challenges. The two main computation modes in GNN inference exhibit distinct characteristics in sparsity and computational density, which results in imbalanced workload distribution and inefficient utilization of compute and memory bandwidth. Furthermore, subgraph partitioning strategies, commonly adopted to address on-chip resource constraints, frequently introduce write-back overhead and port conflicts, thereby limiting overall system throughput. To address these challenges, we propose TernaryGNNs, a high-throughput, area-efficient ternary weight GNN inference framework targeting CPU-FPGA heterogeneous platforms. First, we introduce a precision-preserving ternary quantization method that maintains model accuracy with 1.016% degradation, while achieving an average weight sparsity of 81.1% and a parameter reduction rate of 94.71%. Next, by exploiting sparsity, we reformulate GNN inference as Sparse Matrix–Dense Matrix Multiplication (SpMM) or Sparse General Matrix–Matrix Multiplication (SpGEMM) computation and propose a unified sparse-optimized processor architecture. Finally, we present a comprehensive software–hardware co-design framework that ensures adaptability to the evolving landscape of diverse GNN model architectures. Our framework supports nine mainstream GNN models. Compared to the State-of-the-Art (SOTA) general-purpose GNN processor GraphOPU, TernaryGNNs achieves an average \(2.79\times\) hardware performance improvement, \(1.70\times\) end-to-end performance improvement, and \(2.83\times\) area-efficiency improvement. Compared to the leading overlay accelerator FP-GNN, it delivers \(6.73\times\) hardware performance improvement on average.

Graph neural networks (GNNs) with missing node features have recently received increasing interest. Such missing node features seriously hurt the performance of the existing GNNs. Some recent methods have been proposed to reconstruct the missing node features by the information propagation among nodes with known and unknown attributes. Although these methods have achieved superior performance, how to exactly exploit the complex data correlations among nodes to reconstruct missing node features is still a great challenge. To solve the above problem, we propose a self-supervised guided hypergraph feature propagation (SGHFP). Specifically, the feature hypergraph is first generated according to the node features with missing information. And then, the reconstructed node features produced by the previous iteration are fed to a two-layer GNNs to construct a pseudo-label hypergraph. Before each iteration, the constructed feature hypergraph and pseudo-label hypergraph are fused effectively, which can better preserve the higher-order data correlations among nodes. After then, we apply the fused hypergraph to the feature propagation for reconstructing missing features. Finally, the reconstructed node features by multi-iteration optimization are applied to the downstream semi-supervised classification task. Extensive experiments demonstrate that the proposed SGHFP outperforms the existing semi-supervised classification with missing node feature methods.

Graph Neural Networks (GNNs) have demonstrated significant success in various graph-related tasks. However, their performance is highly dependent on the availability of sufficient, class-balanced annotated data and an accurate graph structure. GNNs are vulnerable to substantial performance degradation when confronted with limited annotated samples, class imbalance, or noisy graph structures. First, a Dual GNN learning framework is introduced for semi-supervised node classification under limited supervision. The framework comprises two GNN-based node prediction modules: a primary module, which uses the input graph structure to induce standard node embeddings and predictions, and an auxiliary module, which leverages spectral clustering to construct a new graph structure and learn new node embeddings and predictions. These modules collaborate to enable end-to-end learning of discriminative node representations. Next, a graph augmentation method called Graph Dual Mixup (GDM) is proposed for graph classification with scarce labels. GDM employs a graph structural auto-encoder to learn structural embeddings, applying mixup in the learned structural embedding space to generate new graph structures. Additionally, mixup is applied to input node features, generating node features for new graph instances. Together, these generated node features and graph structures contribute new graphs that increase the size and diversity of the labeled dataset, improving classification performance. To enhance robustness against adversarial attacks on graph structures, an Efficient Low-Rank GNN is introduced. This approach learns robust low-rank sparse graph structures through a two-stage process. In the first stage, Singular Value Decomposition is used to estimate a low-rank approximation of the graph structure. This estimate is refined in the second stage by jointly learning a low-rank sparse graph structure alongside the GNN model, resulting in enhanced robustness. Finally, to address class-imbalanced node classification, a Unified GNN Learning (Uni-GNN) framework is proposed. Uni-GNN integrates structural and semantic connectivity representations to extend the propagation of embeddings to non-adjacent structural neighbors and semantically similar nodes, facilitating the diffusion of discriminative information. Additionally, it incorporates a balanced pseudo-label generation mechanism to improve the representations of minority classes. Comprehensive evaluations on benchmark datasets demonstrate that these methods outperform state-of-the-art approaches and hold significant potential for advancing GNN learning with limited supervision.

Network representation learning is receiving increasing attention from scholars. Among them, methods based on graph neural networks have become particularly popular. However, most existing methods currently only focus on networks with a single type of relations. In the real world, networks contain a wealth of diverse information, with multiple types of relationships between nodes. In this paper, we propose a graph neural network-based multiplex network representation learning model (GNMRL). We model nodes within each layer of the multiplex network by aggregating neighbor information. Additionally, since nodes are influenced by different network layers, we integrate node interaction information across network layers. We conducted systematic experiments on four datasets, and the results show that GNMRL outperforms other comparison methods in both link prediction and node classification tasks.

In recent years, graph convolutional neural networks (GCNs) have become a popular research topic due to their outstanding performance in various complex network data mining tasks. However, current research on graph neural networks lacks understanding of the high-order structural features of networks, focusing mostly on node features and first-order neighbor features. This article proposes two new models, MGAT and MGATv2, by introducing high-order structure motifs that frequently appear in networks and combining them with graph attention mechanisms. By introducing a mixed information matrix based on motifs, the generation process of graph attention coefficients is improved, allowing the model to capture higher-order structural features. Compared with the latest research on various graph neural networks, both MGAT and MGATv2 achieve good results in node classification tasks. Furthermore, through various experimental studies on real datasets, we demonstrate that the introduction of network structural motifs can effectively enhance the expressive power of graph neural networks, indicating that both high-order structural features and attribute features are important components of network feature learning.

The burgeoning volume of graph data presents significant computational challenges in training graph neural networks (GNNs), critically impeding their efficiency in various applications. To tackle this challenge, graph condensation (GC) has emerged as a promising acceleration solution, focusing on the synthesis of a compact yet representative graph for efficiently training GNNs while retaining performance. Despite the potential to promote scalable use of GNNs, existing GC methods are limited to aligning the condensed graph with merely the observed static graph distribution. This limitation significantly restricts the generalization capacity of condensed graphs, particularly in adapting to dynamic distribution changes. In real-world scenarios, however, graphs are dynamic and constantly evolving, with new nodes and edges being continually integrated. Consequently, due to the limited generalization capacity of condensed graphs, applications that employ GC for efficient GNN training end up with sub-optimal GNNs when confronted with evolving graph structures and distributions in dynamic real-world situations. To overcome this issue, we propose open-world graph condensation (OpenGC), a robust GC framework that integrates structure-aware distribution shift to simulate evolving graph patterns and exploit the temporal environments for invariance condensation. This approach is designed to extract temporal invariant patterns from the original graph, thereby enhancing the generalization capabilities of the condensed graph and, subsequently, the GNNs trained on it. Furthermore, to support the periodic re-condensation and expedite condensed graph updating in life-long graph learning, OpenGC reconstructs the sophisticated optimization scheme with kernel ridge regression and non-parametric graph convolution, significantly accelerating the condensation process while ensuring the exact solutions. Extensive experiments on both real-world and synthetic evolving graphs demonstrate that OpenGC outperforms state-of-the-art (SOTA) GC methods in adapting to dynamic changes in open-world graph environments.

Counterfactual learning is emerging as an important paradigm, rooted in causality, which promises to alleviate common issues of graph neural networks (GNNs), such as fairness and interpretability. However, as in many real-world application domains where conducting randomized controlled trials is impractical, one has to rely on available observational (factual) data to detect counterfactuals. In this paper, we introduce and tackle the problem of searching for counterfactual evidences for the GNN-based node classification task. A counterfactual evidence is a pair of nodes such that, regardless they exhibit great similarity both in the features and in their neighborhood subgraph structures, they are classified differently by the GNN. We develop effective and efficient search algorithms and a novel indexing solution that leverages both node features and structural information to identify counterfactual evidences, and generalizes beyond any specific GNN. Through various downstream applications, we demonstrate the potential of counterfactual evidences to enhance fairness and accuracy of GNNs.

Graph, as an important data representation, is ubiquitous in many real world applications ranging from social network analysis to biology. How to correctly and effectively learn and extract information from graph is essential for a large number of machine learning tasks. Graph embedding is a way to transform and encode the data structure in high dimensional and non-euclidean feature space to a low dimensional and structural space, which is easily exploited by other machine learning algorithms. We have witnessed a huge surge of such embedding methods, from statistical approaches to recent deep learning methods such as the graph convolutional networks (GCN). Deep learning approaches usually outperform the traditional methods in most graph learning benchmarks by building an end-to-end learning framework to optimize the loss function directly. However, most of the existing GCN methods can only perform convolution operations with node features, while ignoring the handy information in edge features, such as relations in knowledge graphs. To address this problem, we present CensNet, Convolution with Edge-Node Switching graph neural network, for learning tasks in graph-structured data with both node and edge features. CensNet is a general graph embedding framework, which embeds both nodes and edges to a latent feature space. By using line graph of the original undirected graph, the role of nodes and edges are switched, and two novel graph convolution operations are proposed for feature propagation. Experimental results on real-world academic citation networks and quantum chemistry graphs show that our approach achieves or matches the state-of-the-art performance in four graph learning tasks, including semi-supervised node classification, multi-task graph classification, graph regression, and link prediction.

Graph Neural Networks (GNNs) have achieved remarkable success by extending traditional convolution to learning on non-Euclidean data. The key to the GNNs is adopting the neural message-passing paradigm with two stages: aggregation and update. The current design of GNNs considers the topology information in the aggregation stage. However, in the updating stage, all nodes share the same updating function. The identical updating function treats each node embedding as i.i.d. random variables and thus ignores the implicit relationships between neighborhoods, which limits the capacity of the GNNs. The updating function is usually implemented with a linear transformation followed by a non-linear activation function. To make the updating function topology-aware, we inject the topological information into the non-linear activation function and propose Graph-adaptive Rectified Linear Unit (GReLU), which is a new parametric activation function incorporating the neighborhood information in a novel and efficient way. The parameters of GReLU are obtained from a hyperfunction based on both node features and the corresponding adjacent matrix. To reduce the risk of overfitting and the computational cost, we decompose the hyperfunction as two independent components for nodes and features respectively. We conduct comprehensive experiments to show that our plug-and-play GReLU method is efficient and effective given different GNN backbones and various downstream tasks.

In recent years, Graph Neural Networks (GNNs) such as Graph Convolutional Networks (GCN) have been gaining momentum in machine learning for graph structured data. Although a majority of GNN architectures are designed in a way that is sustainable to node feature attack, they still suffer from performance degradation under the presence of extreme outlier noise in node features. This is because most strategies dealing with noise rely on the Gaussian noise assumption where an intrinsic Euclidean norm regularization is added to the objective. In this paper, we propose a novel GNN module that is more robust to extreme noise with a Laplace noise assumption. Our proposed module can be used alongside with any existing GNN model to improve the robustness. Numerical experiments show the effectiveness of the new module in enhancing the performance for various GNN models including GCN, GAT, APPNP under the case of extreme noise, and show great robustness under various types of noise added into the datasets.