Hierarchical Structures in Human Networks Using a Graph Neural Network-based Unsupervised Learning Framework

With the rapid deployment of graph neural networks (GNNs) based techniques in a wide range of applications such as link prediction, community detection, and node classification, the explainability of GNNs become an indispensable component for predictive and trustworthy decision making. To achieve this goal, some recent works focus on designing explainable GNN models such as GNNExplainer, PGExplainer, and Gem. These GNN explainers have shown remarkable performance in explaining the predictive results from GNNs. Despite their success, the robustness of these explainers is less explored in terms of vulnerabilities of GNN explainers. Graph perturbations such as adversarial attacks can lead to inaccurate explanations and consequently cause catastrophes. Thus, in this paper, we take the first step and strive to explore the robustness of GNN explainers. To be specific, we first define two adversarial attack scenarios—aggressive adversary and conservative adversary to contaminate graph structures. We then investigate the impacts of the poisoned graphs on the explainability of three prevalent GNN explainers with three standard evaluation metrics: \(Fidelity^+\), \(Fidelity^-\), and Sparsity. We conduct experiments on synthetic and real-world datasets and focus on two popular graph mining tasks: node classification and graph classification. Our empirical results suggest that GNN explainers are generally not robust to the adversarial attacks caused by graph structural noises.KeywordsGraph neural networksGNN explainersAdversarial attacksRobustness

Graph Neural Networks (GNNs) have demonstrated significant success in various graph-related tasks. However, their performance is highly dependent on the availability of sufficient, class-balanced annotated data and an accurate graph structure. GNNs are vulnerable to substantial performance degradation when confronted with limited annotated samples, class imbalance, or noisy graph structures. First, a Dual GNN learning framework is introduced for semi-supervised node classification under limited supervision. The framework comprises two GNN-based node prediction modules: a primary module, which uses the input graph structure to induce standard node embeddings and predictions, and an auxiliary module, which leverages spectral clustering to construct a new graph structure and learn new node embeddings and predictions. These modules collaborate to enable end-to-end learning of discriminative node representations. Next, a graph augmentation method called Graph Dual Mixup (GDM) is proposed for graph classification with scarce labels. GDM employs a graph structural auto-encoder to learn structural embeddings, applying mixup in the learned structural embedding space to generate new graph structures. Additionally, mixup is applied to input node features, generating node features for new graph instances. Together, these generated node features and graph structures contribute new graphs that increase the size and diversity of the labeled dataset, improving classification performance. To enhance robustness against adversarial attacks on graph structures, an Efficient Low-Rank GNN is introduced. This approach learns robust low-rank sparse graph structures through a two-stage process. In the first stage, Singular Value Decomposition is used to estimate a low-rank approximation of the graph structure. This estimate is refined in the second stage by jointly learning a low-rank sparse graph structure alongside the GNN model, resulting in enhanced robustness. Finally, to address class-imbalanced node classification, a Unified GNN Learning (Uni-GNN) framework is proposed. Uni-GNN integrates structural and semantic connectivity representations to extend the propagation of embeddings to non-adjacent structural neighbors and semantically similar nodes, facilitating the diffusion of discriminative information. Additionally, it incorporates a balanced pseudo-label generation mechanism to improve the representations of minority classes. Comprehensive evaluations on benchmark datasets demonstrate that these methods outperform state-of-the-art approaches and hold significant potential for advancing GNN learning with limited supervision.

Graph neural networks (GNNs) have recently achieved remarkable success on a variety of graph-related tasks, while such success relies heavily on a given graph structure that may not always be available in real-world applications. To address this problem, graph structure learning (GSL) is emerging as a promising research topic where task-specific graph structure and GNN parameters are jointly learned in an end-to-end unified framework. Despite their great progress, existing approaches mostly focus on the design of similarity metrics or graph construction, but directly default to adopting downstream objectives as supervision, which lacks deep insight into the power of supervision signals. More importantly, these approaches struggle to explain how GSL helps GNNs, and when and why this help fails. In this article, we conduct a systematic experimental evaluation to reveal that GSL and GNNs enjoy consistent optimization goals in terms of improving the graph homophily. Furthermore, we demonstrate theoretically and experimentally that task-specific downstream supervision may be insufficient to support the learning of both graph structure and GNN parameters, especially when the labeled data are extremely limited. Therefore, as a complement to downstream supervision, we propose homophily-enhanced self-supervision for GSL (HES-GSL), a method that provides more supervision for learning an underlying graph structure. A comprehensive experimental study demonstrates that HES-GSL scales well to various datasets and outperforms other leading methods. Our code will be available in https://github.com/LirongWu/Homophily-Enhanced-Self-supervision.

Community detection has gained significant research interest within the data mining field. It involves identifying subsets of nodes with dense internal connections and sparse external connections. Most studies on community detection focus solely on identifying non-overlapping communities in a static graph. However, in practice, communities often overlap, and the structure of the graphs is dynamically evolving. This dynamic nature leads to community changes and poses a significant challenge in detecting overlapping communities on temporal graphs (T-OCD). While graph neural networks have shown great performance in generating node representations for community detection, learning representations that capture temporal graph structures and support overlapping community detection remain an open question. To address these challenges, we present T-OCDIB , a novel approach for T emporal O verlapping C ommunity D etection guided by I nformation B ottleneck. Specifically, we first propose an overlapping community detection approach for static graphs, under the guidance of a community-oriented information bottleneck. This approach allows us to learn discriminative node representations specific to each community, facilitating the detection of overlapping communities. Following this, we extend this method to temporal graphs by presenting a temporal convolution module. This module uses adaptive weight matrices based on evolving graph structures to capture temporal dependencies for community detection. Additionally, to promote smooth transitions between consecutive communities, we introduce a temporal smoothing module to further constrain changes in community structure. We evaluate the proposed approach on both real-world and synthetic temporal networks. Experimental results illustrate the superiority of T-OCDIB over other community detection methods.

This study presents a novel approach that synergizes community detection algorithms with various Graph Neural Network (GNN) models to bolster link prediction in scientific literature networks. By integrating the Louvain community detection algorithm into our GNN frameworks, we consistently enhanced the performance across all models tested. For example, integrating the Louvain model with the GAT model resulted in an AUC score increase from 0.777 to 0.823, exemplifying the typical improvements observed. Similar gains were noted when the Louvain model was paired with other GNN architectures, confirming the robustness and effectiveness of incorporating community-level insights. This consistent increase in performance—reflected in our extensive experimentation on bipartite graphs of scientific collaborations and citations—highlights the synergistic potential of combining community detection with GNNs to overcome common link prediction challenges such as scalability and resolution limits. Our findings advocate for the integration of community structures as a significant step forward in the predictive accuracy of network science models, offering a comprehensive understanding of scientific collaboration patterns through the lens of advanced machine learning techniques.

Network representation learning is receiving increasing attention from scholars. Among them, methods based on graph neural networks have become particularly popular. However, most existing methods currently only focus on networks with a single type of relations. In the real world, networks contain a wealth of diverse information, with multiple types of relationships between nodes. In this paper, we propose a graph neural network-based multiplex network representation learning model (GNMRL). We model nodes within each layer of the multiplex network by aggregating neighbor information. Additionally, since nodes are influenced by different network layers, we integrate node interaction information across network layers. We conducted systematic experiments on four datasets, and the results show that GNMRL outperforms other comparison methods in both link prediction and node classification tasks.

Graphs are essential representations of many real-world data such as social networks. Recent years have witnessed the increasing efforts made to extend the neural network models to graph-structured data. These methods, which are usually known as the graph neural networks, have been applied to advance many graphs related tasks such as reasoning dynamics of the physical system, graph classification, and node classification. Most of the existing graph neural network models have been designed for static graphs, while many real-world graphs are inherently dynamic. For example, social networks are naturally evolving as new users joining and new relations being created. Current graph neural network models cannot utilize the dynamic information in dynamic graphs. However, the dynamic information has been proven to enhance the performance of many graph analytic tasks such as community detection and link prediction. Hence, it is necessary to design dedicated graph neural networks for dynamic graphs. In this paper, we propose DGNN, a new {\bf D}ynamic {\bf G}raph {\bf N}eural {\bf N}etwork model, which can model the dynamic information as the graph evolving. In particular, the proposed framework can keep updating node information by capturing the sequential information of edges (interactions), the time intervals between edges and information propagation coherently. Experimental results on various dynamic graphs demonstrate the effectiveness of the proposed framework.

Semi-supervised node classification on graph-structured data has many applications such as fraud detection, fake account and review detection, user’s private attribute inference in social networks, and community detection. Various methods such as pairwise Markov Random Fields (pMRF) and graph neural networks were developed for semi-supervised node classification. pMRF is more efficient than graph neural networks. However, existing pMRF-based methods are less accurate than graph neural networks, due to a key limitation that they assume a heuristics-based constant edge potential for all edges. In this work, we aim to address the key limitation of existing pMRF-based methods. In particular, we propose to learn edge potentials for pMRF. Our evaluation results on various types of graph datasets show that our optimized pMRF-based method consistently outperforms existing graph neural networks in terms of both accuracy and efficiency. Our results highlight that previous work may have underestimated the power of pMRF for semi-supervised node classification.

At present, Graph Neural Network (GNN) methods usually follow the node centered message passing process and rely heavily on smooth node characteristics rather than graph structure. In view of this limitation, based on the heuristic method and graph attention mechanism, a feature fusion link prediction model (SAFL) combined with graph neural network is proposed. This model extracts the enclosing subgraphs around the target, combines the attention mechanism to assign neighbor weights to learn useful structural features, considers the impact of different nodes on the link, and fuses the graph neural network with the characteristics of input nodes to predict the link. The experiment on OGB dataset shows that the proposed link prediction model based on heuristic method enhances the graph structure characteristics, effectively represents the connectivity of enclosing subgraphs, and achieves better performance in link prediction.

In recent years, heterogeneous network/graph representation learning/embedding (HNE) has drawn tremendous attentions from research communities in multiple disciplines. HNE has shown its outstanding performances in various networked data analysis and mining tasks. In fact, most of real-world information networks in multiple fields can be modelled as the heterogeneous information networks (HIN). Thus, the HNE-based techniques can sufficiently capture rich-structured and semantic latent features from the given information network in order to facilitate for different task-driven learning tasks. This is considered as fundamental success of HNE-based approach in comparing with previous traditional homogeneous network/graph based embedding techniques. However, there are recent studies have also demonstrated that the heterogeneous network/graph modelling and embedding through graph neural network (GNN) is not usually reliable. This challenge is original come from the fact that most of real-world heterogeneous networks are considered as incomplete and normally contain a large number of feature noises. Therefore, multiple attempts have proposed recently to overcome this limitation. Within this approach, the meta-path-based heterogeneous graph-structured latent features and GNN-based parameters are jointly learnt and optimized during the embedding process. However, this integrated GNN and heterogeneous graph structure (HGS) learning approach still suffered a challenge of effectively parameterizing and fusing different graph-structured latent features from both GNN- and HGS-based sides into better task-driven friendly and noise-reduced embedding spaces. Therefore, in this paper we proposed a novel attention-supplemented heterogeneous graph structure embedding approach, called as: AGSE. Our proposed AGSE model supports to not only achieve the combined rich heterogeneous structural and GNN-based aggregated node representations but also transform achieved node embeddings into noise-reduced and task-driven friendly embedding space. Extensive experiments in benchmark heterogeneous networked datasets for node classification task showed the effectiveness of our proposed AGSE model in comparing with state-of-the-art network embedding baselines.

Graph Neural Networks (GNNs) have become a prominent approach to machine learning with graphs and have been increasingly applied in a multitude of domains. Nevertheless, since most existing GNN models are based on flat message-passing mechanisms, two limitations need to be tackled: (i) they are costly in encoding long-range information spanning the graph structure; (ii) they are failing to encode features in the high-order neighbourhood in the graphs as they only perform information aggregation across the observed edges in the original graph. To deal with these two issues, we propose a novel Hierarchical Message-passing Graph Neural Networks framework. The key idea is generating a hierarchical structure that re-organises all nodes in a flat graph into multi-level super graphs, along with innovative intra- and inter-level propagation manners. The derived hierarchy creates shortcuts connecting far-away nodes so that informative long-range interactions can be efficiently accessed via message passing and incorporates meso- and macro-level semantics into the learned node representations. We present the first model to implement this framework, termed Hierarchical Community-aware Graph Neural Network (HC-GNN), with the assistance of a hierarchical community detection algorithm. The theoretical analysis illustrates HC-GNN’s remarkable capacity in capturing long-range information without introducing heavy additional computation complexity. Empirical experiments conducted on 9 datasets under transductive, inductive, and few-shot settings exhibit that HC-GNN can outperform state-of-the-art GNN models in network analysis tasks, including node classification, link prediction, and community detection. Moreover, the model analysis further demonstrates HC-GNN’s robustness facing graph sparsity and the flexibility in incorporating different GNN encoders.

This study introduces a novel approach to drug discovery in regenerative medicine through the utilization of a graph neural network (GNN). The research methodology integrates the development and training of the GNN with a subsequent evaluation of its performance metrics. The first phase involves the generation of synthetic data simulating a biological network, employing networkX and NumPy libraries to construct a random graph with Erdos-Renyi topology. The data, representing cellular responses to biomaterials, is then converted into PyTorch tensors for compatibility with the GNN architecture. The GNN model, characterized by two fully connected layers with ReLU and log-softmax activations, captures intricate relationships within the graph-structured data. The second phase employs a stochastic gradient descent algorithm, specifically the Adam optimizer, to train the GNN over 100 epochs using the cross-entropy loss for multi-class classification. The research methodology extends to the evaluation phase, producing three distinct output graphs for analysis: Visualization of the graph structure, a comparison between predicted and true labels, and a plot illustrating training loss over epochs. Performance metrics, including accuracy, precision, recall, and F1-score, are computed to assess the model’s predictive capabilities quantitatively. The study concludes with a discussion on the nuances revealed by each graph and their implications for refining GNN models in the context of drug discovery for regenerative medicine.

Graph neural networks (GNNs) have recently gained much attention for node and graph classification tasks on graph-structured data. However, multiple recent works showed that an attacker can easily make GNNs predict incorrectly via perturbing the graph structure, i.e., adding or deleting edges in the graph. We aim to defend against such attacks via developing certifiably robust GNNs. Specifically, we prove the first certified robustness guarantee of any GNN for both node and graph classifications against structural perturbation. Moreover, we show that our certified robustness guarantee is tight. Our results are based on a recently proposed technique called randomized smoothing, which we extend to graph data. We also empirically evaluate our method for both node and graph classifications on multiple GNNs and multiple benchmark datasets. For instance, on the Cora dataset, Graph Convolutional Network with our randomized smoothing can achieve a certified accuracy of 0.49 when the attacker can arbitrarily add/delete at most 15 edges in the graph.

Graph Neural Networks (GNNs) have achieved remarkable success in graph-related tasks by combining node features and graph topology elegantly. Most GNNs assume that the networks are homophilous, which is not always true in the real world, i.e., structure noise or disassortative graphs. Only a few works focus on generalizing graph neural networks to heterophilous or low homophilous networks, where connected nodes may have different labels. In this paper, we design a simple and effective Graph Structure Learning strategy based on Feature and Label consistency (GSLFL) to increase the homophilous level of networks for generalizing any existing GNNs to heterophilous networks. Specifically, we first introduce a method to learn graph structure based on node features and then modify the graph structure based on label consistency. Further, we combine the GSLFL with three existing GNNs to learn node representations and graph structure together. And we design a self-training method to iteratively train models and modify graph structure with pseudo-labels. Finally, our empirical results on 6 public networks with homophily or heterophily, and structure attacks show that our methods outperform the state-of-the-art methods in most cases.

Recent studies have shown that the problems of cold processing and data sparsity in recommender systems can be well alleviated with the help of knowledge graph information. Meanwhile, since the information in recommender systems can be represented by graph data structure, graph neural network has great advantages in recommendatory system. Therefore, graph neural network can better mine the deep relationship between objects in the recommender systems which combines with information from knowledge graph. This paper reviews the application of graph neural network in knowledge graph aided recommendation systems. Specifically, this paper firstly sorts out the related knowledge of knowledge graph, recommendation system and graph neural network, including the cold processing and data sparsity, the structure of knowledge graph, and the basic modules of graph neural network. This paper also summarizes the methods such as KGAT and CKAN used in knowledge graph aided recommendation systems in recent years, and gives some discussion on the development of this field.