Graph Representation Learning Based on Cognitive Spreading Activations

Graph contrastive learning has been developed to learn discriminative node representations on homogeneous graphs. However, it is not clear how to augment the heterogeneous graphs without substantially altering the underlying semantics or how to design appropriate pretext tasks to fully capture the rich semantics preserved in heterogeneous information networks (HINs). Moreover, early investigations demonstrate that contrastive learning suffer from sampling bias, whereas conventional debiasing techniques (e.g., hard negative mining) are empirically shown to be inadequate for graph contrastive learning. How to mitigate the sampling bias on heterogeneous graphs is another important yet neglected problem. To address the aforementioned challenges, we propose a novel multi-view heterogeneous graph contrastive learning framework in this paper. We use metapaths, each of which depicts a complementary element of HINs, as the augmentation to generate multiple subgraphs (i.e., multi-views), and propose a novel pretext task to maximize the coherence between each pair of metapath-induced views. Furthermore, we employ a positive sampling strategy to explicitly select hard positives by jointly considering semantics and structures preserved on each metapath view to alleviate the sampling bias. Extensive experiments demonstrate MCL consistently outperforms state-of-the-art baselines on five real-world benchmark datasets and even its supervised counterparts in some settings.

Recently, graph neural networks (GNNs) have been improved under the influence of various deep learning techniques, such as attention, autoencoders, and recurrent networks. However, real-world graphs may have multiple types of vertices and edges, such as graphs of social networks, citation networks, and e-commerce data. In these cases, most GNNs that consider a homogeneous graph as input data are not suitable because they ignore the heterogeneity. Meta-path-based methods have been researched to capture both heterogeneity and structural information of heterogeneous graphs. As a meta-path is a type of graph pattern, we extend the use of meta-paths to exploit graph patterns. In this study, we propose TP-HAN, a heterogeneous graph attention network for exploiting triangle patterns. In the experiments using DBLP and IMDB, we show that TP-HAN outperforms the state-of-the-art heterogeneous graph attention network.KeywordsGraph neural networksHeterogeneous graphTriangle

Real-world graphs or networks are usually heterogeneous, involving multiple types of nodes and relationships. Heterogeneous graph neural networks (HGNNs) can effectively handle these diverse nodes and edges, capturing heterogeneous information within the graph, thus exhibiting outstanding performance. However, most methods of HGNNs usually involve complex structural designs, leading to problems such as high memory usage, long inference time, and extensive consumption of computing resources. These limitations pose certain challenges for the practical application of HGNNs, especially for resource-constrained devices. To mitigate this issue, we propose the Spiking Heterogeneous Graph Attention Networks (SpikingHAN), which incorporates the brain-inspired and energy-saving properties of Spiking Neural Networks (SNNs) into heterogeneous graph learning to reduce the computing cost without compromising the performance. Specifically, SpikingHAN aggregates metapath-based neighbor information using a single-layer graph convolution with shared parameters. It then employs a semantic-level attention mechanism to capture the importance of different meta-paths and performs semantic aggregation. Finally, it encodes the heterogeneous information into a spike sequence through SNNs, simulating bioinformatic processing to derive a binarized 1-bit representation of the heterogeneous graph. Comprehensive experimental results from three real-world heterogeneous graph datasets show that SpikingHAN delivers competitive node classification performance. It achieves this with fewer parameters, quicker inference, reduced memory usage, and lower energy consumption.

Graph mining tasks often suffer from the lack of supervision from labeled information due to the intrinsic sparseness of graphs and the high cost of manual annotation. To alleviate this issue, inspired by recent advances of self-supervised learning (SSL) on computer vision and natural language processing, graph self-supervised learning methods have been proposed and achieved remarkable performance by utilizing unlabeled information. However, most existing graph SSL methods focus on homogeneous graphs, ignoring the ubiquitous heterogeneity of real-world graphs where nodes and edges are of multiple types. Therefore, directly applying existing graph SSL methods to heterogeneous graphs can not fully capture the rich semantics and their correlations in heterogeneous graphs. In light of this, we investigate self-supervised learning on heterogeneous graphs and propose a novel model named Multi-View Self-supervised heterogeneous graph Embedding (MVSE). By encoding information from different views defined by meta-paths and optimizing both intra-view and inter-view contrastive learning tasks, MVSE comprehensively utilizes unlabeled information and learns node embeddings. Extensive experiments are conducted on various tasks to show the effectiveness of the proposed framework.

To address the challenges faced by existing artificial intelligence methods in modeling complex heterogeneous biological networks, particularly their limitations in capturing collaborative relationships between nodes and in extracting high-order topological semantic features, we propose a novel drug repositioning prediction method based on dynamic representation learning on heterogeneous graphs. A heterogeneous biological graph that integrates drugs, diseases, and their interaction relationships was constructed, based on which a dynamic gated attention module was designed to extract discriminative topological features of drugs and diseases by incorporating a dynamic graph attention mechanism. A gated residual feature fusion mechanism was developed to precisely integrate structural and semantic information from multiple similarity networks to reduce feature redundancy and information loss, thereby enabling accurate prediction of drug-disease associations. Experiments and case studies conducted on multiple drug datasets related to complex diseases demonstrated that the proposed method outperformed existing mainstream models in drug repositioning prediction. The proposed method can effectively model complex associations in heterogeneous biological networks, enhance the accuracy of drug repositioning prediction, and provide important technical support for precision treatment of complex diseases and development of medical artificial intelligence.

Graph representation learning has become a ubiquitous component in many scenarios, ranging from social network analysis to energy forecasting in smart grids. In several applications, ensuring the fairness of the node (or graph) representations with respect to some protected attributes is crucial for their correct deployment. Yet, fairness in graph deep learning remains underexplored, with few solutions available. In particular, the tendency of similar nodes to cluster on several real-world graphs (i.e., homophily) can dramatically worsen the fairness of these procedures. In this article, we propose a novel biased edge dropout algorithm (FairDrop) to counter-act homophily and improve fairness in graph representation learning. FairDrop can be plugged in easily on many existing algorithms, is efficient, adaptable, and can be combined with other fairness-inducing solutions. After describing the general algorithm, we demonstrate its application on two benchmark tasks, specifically, as a random walk model for producing node embeddings, and to a graph convolutional network for link prediction. We prove that the proposed algorithm can successfully improve the fairness of all models with only a small or negligible drop in accuracy, and compares favourably with existing state-of-the-art solutions. In an ablation study, we demonstrate that our algorithm can flexibly interpolate between biasing towards fairness and an unbiased edge dropout. Furthermore, to better evaluate the gains, we propose a new dyadic group definition to measure the bias of a link prediction task when paired with group-based fairness metrics. In particular, we extend the metric used to measure the bias in node embeddings by taking account of the graph structure.

DyHDGE: Dynamic heterogeneous transaction graph embedding for safety-centric fraud detection in financial scenarios

Graphs are integral to model real-world complex systems like social networks, citation networks, transaction networks etc. Real-world graphs are mostly heterogeneous, continuously evolving dynamic graphs (i.e. Heterogeneous continuous-time dynamic graph - HCTDG). Modeling HCTDGs requires effective representation learning, which is difficult because of their entangled structural and temporal dependencies. We introduce HT-Graph, a novel framework aimed to improve link prediction in HCTDG graphs. HT-Graph addresses scalability and computational inefficiencies while focusing on enhancing link prediction accuracy. As formation of new links between nodes depends on their neighborhood, HT-Graph introduces a neighbor-aware memory module (i.e. memory module with a neighbor-store) that stores and updates local neighborhood information of each node efficiently for faster calculation of structural information, eliminating redundant computations required for traditional neighborhood sampling. To introduce parallelism, we used a neighbor-aware restarter to restart the training at any timestamp using interaction history. During training, the restarter module resets memory states at multiple timestamps and learns to mimic the encoder through the knowledge distillation process. This eliminates sequential dependencies, enabling HT-Graph to capture temporal dynamics and structural heterogeneity while ensuring scalability. HT-Graph outperforms state-of-the-art models in link prediction, providing higher scalability, efficiency, and better predictive performance even with limited data.

Aspect-based sentiment analysis aims to predict sentiment polarities of given aspects in text. Most current approaches employ attention-based neural methods to capture semantic relationships between aspects and words in one sentence. However, these methods ignore the fact that sentences with the same aspect and sentiment polarity often share the structure and semantic information in a domain, which leads to lower model performance. To mitigate this problem, we propose a heterogeneous aspect graph neural network (HAGNN) to learn the structure and semantic knowledge from intersentence relationships. Our model is a heterogeneous graph neural network since it contains three different kinds of nodes: word nodes, aspect nodes, and sentence nodes. These nodes can pass structure and semantic information between each other and update their embeddings to improve the performance of our model. To the best of our knowledge, we are the first to use a heterogeneous graph to capture relationships between sentences and aspects. The experimental results on five public datasets show the effectiveness of our model outperforming some state-of-the-art models.

In this paper, we design and evaluate a new substructure-aware Graph Representation Learning (GRL) approach. GRL aims to map graph structure information into low-dimensional representations. While extensive efforts have been made for modeling global and/or local structure information, GRL can be improved by substructure information. Some recent studies exploit adversarial learning to incorporate substructure awareness, but hindered by unstable convergence. This study will address the major research question: is there a better way to integrate substructure awareness into GRL? As subsets of the graph structure, interested substructures (i.e., subgraph) are unique and representative for differentiating graphs, leading to the high correlation between the representation of the graph-level structure and substructures. Since mutual information (MI) is to evaluate the mutual dependence between two variables, we develop a MI inducted substructure-aware GRL method. We decompose the GRL pipeline into two stages: (1) node-level, where we introduce to maximize MI between the original and learned representation by the intuition that the original and learned representation should be highly correlated; (2) graph-level, where we preserve substructures by maximizing MI between the graph-level structure and substructure representation. Finally, we present extensive experimental results to demonstrate the improved performances of our method with real-world data.

Given the non-Euclidean topology inherent in electroencephalogram (EEG) electrode configurations, graph-based approaches, particularly graph neural networks, have shown notable success across diverse EEG classification tasks. However, since the cerebral cortex lobes function individually and/or collaboratively across diverse tasks, there exist substantial differences between intra-lobe and inter-lobe brain intrinsic functional connectivity. Existing graph networks for EEG classification are based on homogeneous graphs, yet the nature of the cerebral cortex aligns more closely with a heterogeneous graph structure. To this end, we propose HeteroEEG for EEG classification, which to the best of our knowledge is the first to reframe the challenge of exploring EEG spatial information, especially decoupling different types of brain lobes and functional connections, as heterogeneous graph reasoning. Specifically, HeteroEEG is designed to be a dual-branch network aware of spatial, spectral, and temporal EEG features. Experimental results justify the superiority of HeteroEEG in pain and emotion recognition compared with other state-of-the-art studies. The heterogeneous graph construction of HeteroEEG may shed light on future graph-based EEG classification network design.

Heterogeneous graph neural networks (HGNNs) have the powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations. Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) designed for homogeneous graphs, especially the attention mechanism and the multi-layer structure. These mechanisms bring excessive complexity, but seldom work studies whether they are really effective on heterogeneous graphs. In this paper, we conduct an in-depth and detailed study of these mechanisms and propose the Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN). To easily capture structural information, SeHGNN pre-computes the neighbor aggregation using a light-weight mean aggregator, which reduces complexity by removing overused neighbor attention and avoiding repeated neighbor aggregation in every training epoch. To better utilize semantic information, SeHGNN adopts the single-layer structure with long metapaths to extend the receptive field, as well as a transformer-based semantic fusion module to fuse features from different metapaths. As a result, SeHGNN exhibits the characteristics of a simple network structure, high prediction accuracy, and fast training speed. Extensive experiments on five real-world heterogeneous graphs demonstrate the superiority of SeHGNN over the state-of-the-arts on both accuracy and training speed.

Graph neural networks empowered by the Transformer’s self-attention mechanism have arisen as a preferred solution for many graph classification and prediction tasks. Despite their efficacy, these networks are often hampered by their quadratic computational complexity and large model size, which pose significant challenges during graph training and inference. In this study, we present an innovative approach to heterogeneous graph transformation that adeptly navigates these limitations by capturing the rich diversity and semantic depth of graphs with various node and edge types. Our method, which streamlines the key–value interaction to a straightforward linear layer operation, maintains the same level of ranking accuracy while significantly reducing computational overhead and accelerating model training. We introduce the “EHG” model, a testament to our approach’s efficacy, showcasing remarkable performance in multiclass node classification on heterogeneous graphs. Our model’s evaluation on the DBLP, ACM, OGBN-MAG, and OAG datasets reveals its superiority over existing heterogeneous graph models under identical hyperparameter configurations. Notably, our model achieves a reduction of approximately 25% in parameter count and nearly 20% savings in training time compared to the leading heterogeneous graph-transformer models.

Graph theory is one of the most significant ideas. It has wide applications in our daily lives. Graph coloring is one of the concepts of graph theory. On the basis of the basics of requirements, different coloring methods are used. In deconstructing avionics, airport connectivity is the most important and crucial component. It has been used in various studies to see how showcase development affects air terminal improvement and how to arrange change so that it is recognized with showcase development. Several graph coloring methods have been developed in graph theory. In this paper, through the use of graph coloring shows the connectivity between domestic and international airports in India. In India, the total number of airports is 134, and 64 have domestic status, while 22 others have international status. This paper proposes an algorithm that connects the domestic and international airports. This paper uses the real-world graph and maps as well as a case study to set up its adequacy and constraints. This paper examines many academics’ publications and methodologies for map design and airport connection networks.

Graph Neural Networks (GNNs) are the current state-of-the-art models in learning node representations for many predictive tasks on graphs. Typically, GNNs reuses the same set of model parameters across all nodes in the graph to improve the training efficiency and exploit the translationally-invariant properties in many datasets. However, the parameter sharing scheme prevents GNNs from distinguishing two nodes that are isomorphic and that the translation invariance property may not exhibit in real-world graphs. In this paper, we present Graph Representation Learning with Adaptive Mixtures (GRAM), a novel approach for learning node representations in a graph by introducing multiple independent GNN models and a trainable mixture distribution for each node. GRAM is a general framework that can be readily embedded into existing models for enhancing representation learning. It allows for adaptive node clustering with mixtures and joint optimization of all GNN models simultaneously. To achieve this, we develop an Expectation-Maximization algorithm with stochastic gradient descent in a scalable manner. Specifically, in the E-Step, we approximate the posterior probability distribution of the latent cluster membership based on the k-th independent GNN models in the last iteration. In the M-step, we update all GNN models and the mixture parameters based on the refined latent cluster membership for downstream tasks, e.g., node classification. To fully exploit the graph structural information, we further implement a regularization term on unlabeled nodes, which encourages adjacent nodes to have the same mixture distribution. By jointly optimizing the model in an end-to-end manner, GRAM learns to capture different abstract patterns on graphs for different node clusters, which yields discriminative representation for each node. We evaluate GRAM on five benchmark datasets with extensive experiments. GRAM is demonstrated to consistently boost state-of-the-art GNN variants in node classification tasks.