Graph Attention Networks with Positional Embeddings

Graph neural networks (GNNs) typically exhibit superior performance in shallow architectures. However, as the network depth increases, issues such as overfitting and oversmoothing of hidden vector representations arise, significantly diminishing model performance. To address these challenges, this paper proposes a Two-Channel Classification Algorithm Based on Graph Attention Network (TCC_GAT). Initially, nodes exhibiting similar interaction behaviors are identified through cosine similarity, thereby enhancing the foundational graph structure. Subsequently, an attention mechanism is employed to adaptively integrate neighborhood information within the enhanced graph structure, with a multi-head attention mechanism applied to mitigate overfitting. Furthermore, the K-nearest neighbors algorithm is adopted to reconstruct the basic graph structure, facilitating the learning of structural information and neighborhood features that are challenging to capture on interaction graphs. This approach addresses the difficulties associated with learning high-order neighborhood information. Finally, the embedding representations of identical nodes across different graph structures are fused to optimize model classification performance, significantly enhancing node embedding representations and effectively alleviating the over-smoothing issue. Semi-supervised experiments and ablation studies conducted on the Cora, Citeseer, and Pubmed datasets reveal an accuracy improvement ranging from 1.4% to 4.5% compared to existing node classification algorithms. The experimental outcomes demonstrate that the proposed TCC_GAT achieves superior classification results in node classification tasks.

Graph Neural Networks (GNNs) are deep learning models specifically designed for analyzing graph-structured data, capturing complex relationships and structures to improve analysis and prediction. A common task in GNNs is node classification, where each node in the graph is assigned a predefined category. The Graph Attention Network (GAT) is a popular variant of GNNs known for its ability to capture complex dependencies by assigning importance weights to nodes during information aggregation. However, the GAT’s reliance on local attention mechanisms limits its effectiveness in capturing global information and long-range dependencies. To address this limitation, we propose a new attention mechanism called Global-Local Graph Attention (GLGA). Our mechanism enables the GAT to capture long-range dependencies and global graph structures while maintaining its ability to focus on local interactions. We evaluate our algorithm on three citation datasets (Cora, Citeseer, and Pubmed) using multiple metrics, demonstrating its superiority over other baseline models. The proposed GLGA mechanism has been proven to be an effective solution for improving node classification tasks.

Graph neural networks (GNNs) is an information - processing system that uses message passing among graph nodes. In recent years, GNN variants including graph attention network (GAT), graph convolutional network (GCN), and graph recurrent network (GRN) have shown revolutionary performance in computer vision applications using deep learning and artificial intelligence. These neural network model extensions, collect information in the form of graphs. GNN may be divided into three groups based on the challenges it solves: link prediction, node classification, graph classification. Machines can differentiate and recognise objects in image and video using standard CNNs. Extensive amount of research work needs to be done before robots can have same visual intuition as humans. GNN architectures, on the other hand, may be used to solve various image categorization and video challenges. The number of GNN applications in computer vision not limited, continues to expand. Human-object interaction, actin understanding, image categorization from a few shots and many more. In this paper use of GNN in image and video understanding, design aspects, architecture, applications and implementation challenges towards computer vision is described. GNN is a strong tool for analysing graph data and is still a relatively active area that needs further researches attention to solve many computer vision applications.

Graph data is omnipresent and has a wide variety of applications, such as in natural science, social networks, or the semantic web. However, while being rich in information, graphs are often noisy and incomplete. As a result, graph completion tasks, such as node classification or link prediction, have gained attention. On one hand, neural methods, such as graph neural networks, have proven to be robust tools for learning rich representations of noisy graphs. On the other hand, symbolic methods enable exact reasoning on graphs. We propose Knowledge Enhanced Graph Neural Networks (KeGNN), a neuro-symbolic framework for graph completion that combines both paradigms as it allows for the integration of prior knowledge into a graph neural network model. Essentially, KeGNN consists of a graph neural network as a base upon which knowledge enhancement layers are stacked with the goal of refining predictions with respect to prior knowledge. We instantiate KeGNN in conjunction with two well-known graph neural networks, Graph Convolutional Networks and Graph Attention Networks, and evaluate KeGNN on multiple benchmark datasets for node classification.

Network representation learning is receiving increasing attention from scholars. Among them, methods based on graph neural networks have become particularly popular. However, most existing methods currently only focus on networks with a single type of relations. In the real world, networks contain a wealth of diverse information, with multiple types of relationships between nodes. In this paper, we propose a graph neural network-based multiplex network representation learning model (GNMRL). We model nodes within each layer of the multiplex network by aggregating neighbor information. Additionally, since nodes are influenced by different network layers, we integrate node interaction information across network layers. We conducted systematic experiments on four datasets, and the results show that GNMRL outperforms other comparison methods in both link prediction and node classification tasks.

Graph Neural Networks (GNNs) are the current state-of-the-art models in learning node representations for many predictive tasks on graphs. Typically, GNNs reuses the same set of model parameters across all nodes in the graph to improve the training efficiency and exploit the translationally-invariant properties in many datasets. However, the parameter sharing scheme prevents GNNs from distinguishing two nodes that are isomorphic and that the translation invariance property may not exhibit in real-world graphs. In this paper, we present Graph Representation Learning with Adaptive Mixtures (GRAM), a novel approach for learning node representations in a graph by introducing multiple independent GNN models and a trainable mixture distribution for each node. GRAM is a general framework that can be readily embedded into existing models for enhancing representation learning. It allows for adaptive node clustering with mixtures and joint optimization of all GNN models simultaneously. To achieve this, we develop an Expectation-Maximization algorithm with stochastic gradient descent in a scalable manner. Specifically, in the E-Step, we approximate the posterior probability distribution of the latent cluster membership based on the k-th independent GNN models in the last iteration. In the M-step, we update all GNN models and the mixture parameters based on the refined latent cluster membership for downstream tasks, e.g., node classification. To fully exploit the graph structural information, we further implement a regularization term on unlabeled nodes, which encourages adjacent nodes to have the same mixture distribution. By jointly optimizing the model in an end-to-end manner, GRAM learns to capture different abstract patterns on graphs for different node clusters, which yields discriminative representation for each node. We evaluate GRAM on five benchmark datasets with extensive experiments. GRAM is demonstrated to consistently boost state-of-the-art GNN variants in node classification tasks.

Networks are general tools for modeling numerous information with features and complex relations. Network Embedding aims to learn low-dimension representations for vertexes in the network with rich information including content information and structural information. In recent years, many models based on neural network have been proposed to map the network representations into embedding space whose dimension is much lower than that in original space. However, most of existing methods have the following limitations: 1) they are based on content of nodes in network, failing to measure the structure similarity of nodes; 2) they cannot do well in protecting the privacy of users including the original content information and the structural information. In this paper, we propose a similarity-based graph neural network model, SGNN, which captures the structure information of nodes precisely in node classification tasks. It also takes advantage of the thought of federated learning to hide the original information from different data sources to protect users’ privacy. We use deep graph neural network with convolutional layers and dense layers to classify the nodes based on their structures and features. The node classification experiment results on public data sets including Aminer coauthor network, Brazil and Europe flight networks indicate that our proposed model outperforms state-of-the-art models with a higher accuracy.

Graphs help to define the relationships between entities in the data. These relationships, represented by edges, often provide additional context information which can be utilised to discover patterns in the data. Graph Neural Networks (GNNs) employ the inductive bias of the graph structure to learn and predict on various tasks. The primary operation of graph neural networks is the feature aggregation step performed over neighbours of the node based on the structure of the graph. In addition to its own features, for each hop, the node gets additional combined features from its neighbours. These aggregated features help define the similarity or dissimilarity of the nodes with respect to the labels and are useful for tasks like node classification. However, in real‐world data, features of neighbours at different hops may not correlate with the node's features. Thus, any indiscriminate feature aggregation by GNN might cause the addition of noisy features leading to degradation in model's performance. In this work, we show that selective aggregation of node features from various hops leads to better performance than default aggregation on the node classification task. Furthermore, we propose a Dual‐Net GNN architecture with a classifier model and a selector model. The classifier model trains over a subset of input node features to predict node labels while the selector model learns to provide optimal input subset to the classifier for the best performance. These two models are trained jointly to learn the best subset of features that give higher accuracy in node label predictions. With extensive experiments, we show that our proposed model outperforms both feature selection methods and state‐of‐the‐art GNN models with remarkable improvements up to 27.8%.

Recommendation systems is the core of most online experiences, guiding people to the right products, films, music, courses, and even social links when the choices are overwhelming. They work by learning from large interaction logs and applying predictive models to surface content that fits each user’s tastes, steadily improving as algorithms and data grow richer. As AI and deep learning have developed, classic strategies like collaborative filtering and content-based filtering are increasingly paired with, or superseded by, neural architectures that can model more nuanced patterns in behaviour and content. Among these neural approaches, graph neural networks—and especially Graph Attention Networks (GATs)—have become influential for recommendation because they treat users and items as nodes in a graph and learn from the connections between them. With attention mechanisms, GATs decide which neighbours matter most for a given prediction, amplifying useful signals and dampening noise to learn stronger user and item representations. This survey traces how the industry moved from traditional methods to graph-based models centred on GATs, comparing approaches on scalability, ranking accuracy, data sparsity, and interpretability across widely used benchmarks. Drawing from recent peer‑reviewed work, it highlights architectures and training practices that consistently help in real-world settings, as well as gaps that still limit robust deployment at scale. Looking forward, several directions stand out: using explainable AI to clarify why specific items are recommended, combining reinforcement learning with GATs for adaptive policies, and adopting federated learning to protect user privacy while training on distributed data. The versatility of GAT-based recommenders is already visible across domains—from e‑commerce and online learning to healthcare and social feeds—where graph structure and selective attention naturally fit the data. Overall, the evidence points to GATs as a key ingredient for the next generation of personalized, context‑aware, and trustworthy recommendation systems. Keywords: Recommendation Systems, Collaborative Filtering, Graph Neural Networks, Graph Attention Networks

Edge features contain important information about graphs. However, current state-of-the-art neural network models designed for graph learning, e.g. graph convolutional networks (GCN) and graph attention networks (GAT), adequately utilize edge features, especially multi-dimensional edge features. In this paper, we build a new framework for a family of new graph neural network models that can more sufficiently exploit edge features, including those of undirected or multi-dimensional edges. The proposed framework can consolidate current graph neural network models; e.g. graph convolutional networks (GCN) and graph attention networks (GAT). The proposed framework and new models have the following novelties: First, we propose to use doubly stochastic normalization of graph edge features instead of the commonly used row or symmetric normalization approches used in current graph neural networks. Second, we construct new formulas for the operations in each individual layer so that they can handle multi-dimensional edge features. Third, for the proposed new framework, edge features are adaptive across network layers. As a result, our proposed new framework and new models can exploit a rich source of graph information. We apply our new models to graph node classification on several citation networks, whole graph classification, and regression on several molecular datasets. Compared with the current state-of-the-art methods, i.e. GCNs and GAT, our models obtain better performance, which testify to the importance of exploiting edge features in graph neural networks.

The n-octanol/buffer solution distribution coefficient at pH = 7.4 (log D7.4) is an indicator of lipophilicity, and it influences a wide variety of absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties and druggability of compounds. In log D7.4 prediction, graph neural networks (GNNs) can uncover subtle structure-property relationships (SPRs) by automatically extracting features from molecular graphs that facilitate the learning of SPRs, but their performances are often limited by the small size of available datasets. Herein, we present a transfer learning strategy called pretraining on computational data and then fine-tuning on experimental data (PCFE) to fully exploit the predictive potential of GNNs. PCFE works by pretraining a GNN model on 1.71 million computational log D data (low-fidelity data) and then fine-tuning it on 19,155 experimental log D7.4 data (high-fidelity data). The experiments for three GNN architectures (graph convolutional network (GCN), graph attention network (GAT), and Attentive FP) demonstrated the effectiveness of PCFE in improving GNNs for log D7.4 predictions. Moreover, the optimal PCFE-trained GNN model (cx-Attentive FP, Rtest2 = 0.909) outperformed four excellent descriptor-based models (random forest (RF), gradient boosting (GB), support vector machine (SVM), and extreme gradient boosting (XGBoost)). The robustness of the cx-Attentive FP model was also confirmed by evaluating the models with different training data sizes and dataset splitting strategies. Therefore, we developed a webserver and defined the applicability domain for this model. The webserver (http://tools.scbdd.com/chemlogd/) provides free log D7.4 prediction services. In addition, the important descriptors for log D7.4 were detected by the Shapley additive explanations (SHAP) method, and the most relevant substructures of log D7.4 were identified by the attention mechanism. Finally, the matched molecular pair analysis (MMPA) was performed to summarize the contributions of common chemical substituents to log D7.4, including a variety of hydrocarbon groups, halogen groups, heteroatoms, and polar groups. In conclusion, we believe that the cx-Attentive FP model can serve as a reliable tool to predict log D7.4 and hope that pretraining on low-fidelity data can help GNNs make accurate predictions of other endpoints in drug discovery.

Signed attention based graph neural network for graphs with heterophily

AI-Driven molecule generation and bioactivity prediction: A multi-model approach combining VAE, graph and language-based neural networks.

<title>Abstract</title> Rational vaccine design against emerging and understudied pathogens is hindered by incomplete protein–protein interaction (PPI) maps and limited integration of computational prioritization with network-based inference. Here, we present a unified framework that couples Biomni, a biomedical AI agent providing vaccine-priority and virulence-aware annotations, with graph neural network (GNN) based link prediction on STRING-derived PPIs to systematically identify candidate vaccine targets in Flavobacterium. Starting from the complete Flavobacterium proteome, Biomni-derived antigenicity, subcellular, and virulence features were integrated with high-confidence STRING associations into a unified PPI graph in which isolated and weakly connected proteins were explicitly retained. We evaluated three representative GNN architectures Graph Convolutional Network (GCN), Graph Attention Network (GAT), and GraphSAGE under a consistent three-fold cross-validation scheme and performed priorityaware assessment of predicted high-confidence links (posterior probability ≥ 0.90). GraphSAGE achieved the highest best-fold ROC-AUC (0.5585), followed by GAT (0.5536) and GCN (0.5000), reflecting modest yet meaningful discriminative performance in this sparse, single-species setting. Class-pair analyses revealed that GCN predominantly reconstructed links among lower-priority nodes, while GAT modestly increased coverage of interactions involving High-priority proteins but 1 remained biased toward low-tier combinations. In contrast, GraphSAGE produced a more balanced distribution of predicted links across Biomni priority tiers, including enriched High–Medium and High–Low connections, indicating more effective use of antigenicity and virulence features for inductive generalization to under-characterized proteins. Collectively, these results demonstrate that integrating AI-driven prioritization with inductive GNNs enables biologically informed exploration of missing PPIs and highlights previously overlooked Flavobacterium proteins as plausible vaccine candidates. The proposed Biomni–STRING–GNN framework is modular and transferable, offering a principled template for priority-aware, network-based vaccine target discovery in data-sparse pathogen systems.

Technology CAD (TCAD) tools are pivotal for transistor modeling, enabling physics-based simulations with high accuracy essential for developing next-generation technology nodes. However, their high computational cost and low throughput severely constrain large-scale design-space exploration. Accelerating TCAD simulations while preserving physical accuracy has therefore become a central research objective. A promising approach involves the use of graph-based representations of TCAD meshes fed into graph neural networks (GNNs), which offer rapid and accurate alternatives. However, traditional GNNs, such as Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs) often falter with noisy transistor-mesh graphs, where embeddings become cluttered and complex. In thiswork, we propose a self-supervisedGNNframework for TCAD device modeling, specifically employing the Self-Supervised GAT (SuperGAT).We demonstrate that self-supervised GNN models, in contrast to GCNs and GATs, introduces a self-supervised edge-prediction task to distinguish informative from redundant topological connections. Our evaluation compares various GNN aggregation schemes demonstrates that SuperGAT successfully reproduces key transistor transfer characteristics drain current–gate voltage (Id–Vg) curves, subthreshold swing (SS) and threshold voltage (Vth). The dataset was generated using 14nm FDSOI transistors. Our SuperGAT model achieves a minimum <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> score of 0.992 across all characteristics and an inference time of only 0.007 134 sec per transistor, resulting in a speed-up exceeding 100 000 x compared to TCAD simulation, which underscore the capability to substantially accelerate TCAD transistor simulations in advanced technology development.