Enhancing Graph Convolutional Networks by Topology Sampling

Sex classification is a major benchmark of previous work in learning on the structural connectome, a naturally occurring brain graph that has proven useful for studying cognitive function and impairment. While graph neural networks (GNNs), specifically graph convolutional networks (GCNs), have gained popularity lately for their effectiveness in learning on graph data, achieving strong performance in adult sex classification tasks, their application to pediatric populations remains unexplored. We seek to characterize the capacity for GNN models to learn connectomic patterns on pediatric data through an exploration of training techniques and architectural design choices. Two datasets comprising an adult BRIGHT dataset (N = 147 Hodgkin's lymphoma survivors and N = 162 age similar controls) and a pediatric Human Connectome Project in Development (HCP-D) dataset (N = 135 healthy subjects) were utilized. Two GNN models (GCN simple and GCN residual), a deep neural network (multi-layer perceptron), and two standard machine learning models (random forest and support vector machine) were trained. Architecture exploration experiments were conducted to evaluate the impact of network depth, pooling techniques, and skip connections on the ability of GNN models to capture connectomic patterns. Models were assessed across a range of metrics including accuracy, AUC score, and adversarial robustness. GNNs outperformed other models across both populations. Notably, adult GNN models achieved 85.1% accuracy in sex classification on unseen adult participants, consistent with prior studies. The extension of the adult models to the pediatric dataset and training on the smaller pediatric dataset were sub-optimal in their performance. Using adult data to augment pediatric models, the best GNN achieved comparable accuracy across unseen pediatric (83.0%) and adult (81.3%) participants. Adversarial sensitivity experiments showed that the simple GCN remained the most robust to perturbations, followed by the multi-layer perceptron and the residual GCN. These findings underscore the potential of GNNs in advancing our understanding of sex-specific neurological development and disorders and highlight the importance of data augmentation in overcoming challenges associated with small pediatric datasets. Further, they highlight relevant tradeoffs in the design landscape of connectomic GNNs. For example, while the simpler GNN model tested exhibits marginally worse accuracy and AUC scores in comparison to the more complex residual GNN, it demonstrates a higher degree of adversarial robustness.

The n-octanol/buffer solution distribution coefficient at pH = 7.4 (log D7.4) is an indicator of lipophilicity, and it influences a wide variety of absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties and druggability of compounds. In log D7.4 prediction, graph neural networks (GNNs) can uncover subtle structure-property relationships (SPRs) by automatically extracting features from molecular graphs that facilitate the learning of SPRs, but their performances are often limited by the small size of available datasets. Herein, we present a transfer learning strategy called pretraining on computational data and then fine-tuning on experimental data (PCFE) to fully exploit the predictive potential of GNNs. PCFE works by pretraining a GNN model on 1.71 million computational log D data (low-fidelity data) and then fine-tuning it on 19,155 experimental log D7.4 data (high-fidelity data). The experiments for three GNN architectures (graph convolutional network (GCN), graph attention network (GAT), and Attentive FP) demonstrated the effectiveness of PCFE in improving GNNs for log D7.4 predictions. Moreover, the optimal PCFE-trained GNN model (cx-Attentive FP, Rtest2 = 0.909) outperformed four excellent descriptor-based models (random forest (RF), gradient boosting (GB), support vector machine (SVM), and extreme gradient boosting (XGBoost)). The robustness of the cx-Attentive FP model was also confirmed by evaluating the models with different training data sizes and dataset splitting strategies. Therefore, we developed a webserver and defined the applicability domain for this model. The webserver (http://tools.scbdd.com/chemlogd/) provides free log D7.4 prediction services. In addition, the important descriptors for log D7.4 were detected by the Shapley additive explanations (SHAP) method, and the most relevant substructures of log D7.4 were identified by the attention mechanism. Finally, the matched molecular pair analysis (MMPA) was performed to summarize the contributions of common chemical substituents to log D7.4, including a variety of hydrocarbon groups, halogen groups, heteroatoms, and polar groups. In conclusion, we believe that the cx-Attentive FP model can serve as a reliable tool to predict log D7.4 and hope that pretraining on low-fidelity data can help GNNs make accurate predictions of other endpoints in drug discovery.

Graph data is omnipresent and has a wide variety of applications, such as in natural science, social networks, or the semantic web. However, while being rich in information, graphs are often noisy and incomplete. As a result, graph completion tasks, such as node classification or link prediction, have gained attention. On one hand, neural methods, such as graph neural networks, have proven to be robust tools for learning rich representations of noisy graphs. On the other hand, symbolic methods enable exact reasoning on graphs. We propose Knowledge Enhanced Graph Neural Networks (KeGNN), a neuro-symbolic framework for graph completion that combines both paradigms as it allows for the integration of prior knowledge into a graph neural network model. Essentially, KeGNN consists of a graph neural network as a base upon which knowledge enhancement layers are stacked with the goal of refining predictions with respect to prior knowledge. We instantiate KeGNN in conjunction with two well-known graph neural networks, Graph Convolutional Networks and Graph Attention Networks, and evaluate KeGNN on multiple benchmark datasets for node classification.

Graph Neural Networks (GNNs) are the current state-of-the-art models in learning node representations for many predictive tasks on graphs. Typically, GNNs reuses the same set of model parameters across all nodes in the graph to improve the training efficiency and exploit the translationally-invariant properties in many datasets. However, the parameter sharing scheme prevents GNNs from distinguishing two nodes that are isomorphic and that the translation invariance property may not exhibit in real-world graphs. In this paper, we present Graph Representation Learning with Adaptive Mixtures (GRAM), a novel approach for learning node representations in a graph by introducing multiple independent GNN models and a trainable mixture distribution for each node. GRAM is a general framework that can be readily embedded into existing models for enhancing representation learning. It allows for adaptive node clustering with mixtures and joint optimization of all GNN models simultaneously. To achieve this, we develop an Expectation-Maximization algorithm with stochastic gradient descent in a scalable manner. Specifically, in the E-Step, we approximate the posterior probability distribution of the latent cluster membership based on the k-th independent GNN models in the last iteration. In the M-step, we update all GNN models and the mixture parameters based on the refined latent cluster membership for downstream tasks, e.g., node classification. To fully exploit the graph structural information, we further implement a regularization term on unlabeled nodes, which encourages adjacent nodes to have the same mixture distribution. By jointly optimizing the model in an end-to-end manner, GRAM learns to capture different abstract patterns on graphs for different node clusters, which yields discriminative representation for each node. We evaluate GRAM on five benchmark datasets with extensive experiments. GRAM is demonstrated to consistently boost state-of-the-art GNN variants in node classification tasks.

Developing fast and accurate computational models to simulate intricate physical phenomena has been a persistent research challenge. Recent studies have demonstrated remarkable capabilities in predicting various physical outcomes through machine learning-assisted approaches. However, it remains challenging to generalize current methods, usually crafted for a specific problem, to other more complex or broader scenarios. To address this challenge, we developed graph neural network (GNN) models with enhanced generalizability derived from the distinct GNN architecture and neural operator techniques. As a proof of concept, we employ our GNN models to predict finite element (FE) simulation results for three-dimensional solid mechanics problems with varying boundary conditions. Results show that our GNN model achieves accurate and robust performance in predicting the stress and deformation profiles of structures compared with FE simulations. Furthermore, the neural operator embedded GNN approach enables learning and predicting various solid mechanics problems in a generalizable fashion, making it a promising approach for surrogate modeling.

A multi-agent artificial intelligence (AI) model is developed to automate the discovery of new metallic alloys, integrating multimodal data and external knowledge, including insights from physics via atomistic simulations. The system consists of (a) large language models (LLMs) for tasks such as reasoning and planning, (b) AI agents with distinct roles collaborating dynamically, and (c) a newly developed graph neural network (GNN) model for rapid retrieval of physical properties. We chose the ternary NbMoTa body-centered-cubic alloy as our model system and developed the GNN to predict two fundamental materials properties: the Peierls barrier and the solute/screw dislocation interaction energy. Our GNN model efficiently predicts these properties, reducing reliance on costly brute-force calculations and alleviating the computational demands on the multi-agent system. By combining the predictive capabilities of GNNs with the collaborative intelligence of LLM-driven reasoning agents, the system autonomously explores vast alloy design spaces, identifies trends in atomic-scale properties, and predicts macroscale mechanical strength, as demonstrated by several computational experiments. This synergistic approach accelerates the discovery of advanced alloys and holds promise for broader applications in other complex systems, marking a step forward in automated materials discovery and design. Impact statement Traditional deep learning models, such as graph neural networks and convolutional neural networks, operate within the confines of their training data sets, making single-step inferences for regression or classification. Our work introduces a multi-agent strategy that transcends these limitations by integrating deep learning with reasoning and decision-making capabilities. This intelligent system actively interprets results, determines subsequent actions, and iteratively refines predictions, accelerating the materials design process. We demonstrate its effectiveness in exploring the vast compositional space of a ternary alloy, where the model dynamically solicits data, analyzes trends, generates visualizations, and derives insights into materials behavior. By enabling accurate predictions of key alloy characteristics, our approach advances the discovery of novel metallic systems and underscores the critical role of solid-solution alloying. More broadly, it represents a major step toward integrating artificial intelligence with scientific reasoning, moving closer to artificial general intelligence in engineering. This paradigm shift has profound implications for materials science, enabling more efficient, autonomous, and intelligent exploration of complex materials spaces. Graphical Abstract

Past events have revealed that widespread blackouts are mostly a result of cascading failures in the power grid. Understanding the underlining mechanisms of cascading failures can help in developing strategies to minimize the risk of such events. Moreover, a real-time detection of precursors to cascading failures will help operators take measures to prevent their propagation. Currently, the well-established probabilistic and physics-based models of cascading failures offer low computational efficiency, hindering them to be used only as offline tools. In this work, we develop a data-driven methodology for online estimation of the risk of cascading failures. We utilize a physics-based cascading failure model to generate a cascading failure dataset considering different operating conditions and failure scenarios, thus obtaining a sample space covering a large set of power grid states that are labeled as safe or unsafe. We use the synthetic data to train deep learning architectures, namely Feed-forward Neural Networks (FNN) and Graph Neural Networks (GNN). With the development of GNNs, improved performance is achieved with graph-structured data, and GNNs can generalize to graphs of diverse sizes. A comparison between FNN and GNN is made and the GNNs inductive capability is demonstrated via test grids. Furthermore, we apply transfer learning to improve the performance of a pre-trained GNN model on power grids not seen in the training process. The GNN model shows accuracy and balanced accuracy above 96% on selected test datasets not used in the training. Conversely, the FNN shows accuracy above 85% and balanced accuracy above 81% on test datasets unseen during training. Overall, the GNN model is successful in determining, if one or several simultaneous outages result in a critical grid state, under specific grid operating conditions.

Graph neural networks (GNNs) have made rapid developments in the recent years. Due to their great ability in modeling graph-structured data, GNNs are vastly used in various applications, including high-stakes scenarios such as financial analysis, traffic predictions, and drug discovery. Despite their great potential in benefiting humans in the real world, recent study shows that GNNs can leak private information, are vulnerable to adversarial attacks, can inherit and magnify societal bias from training data and lack interpretability, which have risk of causing unintentional harm to the users and society. For example, existing works demonstrate that attackers can fool the GNNs to give the outcome they desire with unnoticeable perturbation on training graph. GNNs trained on social networks may embed the discrimination in their decision process, strengthening the undesirable societal bias. Consequently, trust-worthy GNNs in various aspects are emerging to prevent the harm from GNN models and increase the users’ trust in GNNs. In this paper, we give a comprehensive survey of GNNs in the computational aspects of privacy, robustness, fairness, and explainability. For each aspect, we give the taxonomy of the related methods and formulate the general frameworks for the multiple categories of trustworthy GNNs. We also discuss the future research directions of each aspect and connections between these aspects to help achieve trustworthiness.

Deep learning has led to major advances in fields like natural language processing, computer vision, and other Euclidean data domains. Yet, many important fields have data defined on irregular domains, requiring graphs to be explicitly modeled. One such application is drug discovery. Recently, research has found that using graph neural network (GNN) models, given enough data, can perform better than using human-engineered fingerprints or descriptors in predicting molecular properties of potential antibiotics.We explore these state-of-the-art AI models on predicting desirable molecular properties for drugs that can inhibit SARS-CoV-2. We build upon the GNN models with ideas from recent breakthroughs in geometric deep learning, inspired by the topologies of the molecules. In this poster paper, we present an overview of the drug discovery framework, drug-target interaction framework, and GNNs. Preliminary results on two COVID-19 related datasets are encouraging, achieving a ROC-AUC of 0.72 for FDA-approved chemical library screened against SARS-CoV-2 in vitro.

Graph Neural Networks (GNNs) have become indispensable tools for learning from graph-structured data, catering to various applications such as social network analysis and fraud detection for financial services. At the heart of these networks are the edges, which are crucial in guiding GNN models' predictions. In many scenarios, these edges represent sensitive information, such as personal associations or financial dealings, which require privacy assurance. However, their contributions to GNN model predictions may, in turn, be exploited by the adversary to compromise their privacy. Motivated by these conflicting requirements, this paper investigates edge privacy in contexts where adversaries possess only black-box access to the target GNN model, restricted further by access controls, preventing direct insights into arbitrary node outputs. Moreover, we are the first to extensively examine situations where the target graph continuously evolves—a common trait of many real-world graphs. In this setting, we present a range of attacks that leverage the message-passing mechanism of GNNs. We evaluated the effectiveness of our attacks using nine real-world datasets, encompassing both static and dynamic graphs, across four different GNN architectures. The results demonstrate that our attack outperforms existing methods across various GNN architectures, consistently achieving an F1 score of at least 0.8 in static scenarios. Furthermore, our attack retains robustness in dynamic graph scenarios, maintaining F1 scores up to 0.8, unlike previous methods that only achieve F1 scores around 0.2.

Most distribution system operators may not accurately record or completely maintain the phase connections for the numerous LV customers. Different consumer phase identification (CPI) approaches based on voltages, powers or other measurements are proposed in the literature. Due to the technical challenges in collecting voltage measurements, power measurement based approaches are preferable. Hence, this paper proposes a novel power based CPI methodology applying Graph Neural Networks (GNNs). The CPI methodology generates synthetic transformer power profiles assuming random combinations of phases for the measured load profiles, which are used altogether to train the GNN model. The GNN model is then tested using measured transformer and load power profiles. The performance of the methodology is evaluated in a test low voltage grid of 55 loads under various conditions of Photovoltaic penetration, Photovoltaics under maintenance, measurement errors, unmetered consumption, uncertain grid asset parameters and inaccurate phase connections. Further tests on a real low voltage grid with 111 loads prove the scalability of the methodology. The attained results show that the GNN model can achieve accuracy above 90% in most cases, outperforming various state-of-the-art methods. • The Graph Neural Network identifies the phase labels with accuracy higher than 90%. • Under noisy or unmetered demand, the Graph Neural Network achieves accuracy over 90%. • Under various Photovoltaic cases, the Graph Neural Network yields accuracy over 90%. • The Graph Neural Network yields accuracy of 92% in case of multi-phase connections. • The grid losses and uncertain assets’ parameters have minor impact on model accuracy.

Traditional Graph Neural Network (GNN) learning patterns can only achieve optimal performance under the assumption of independent and identically distributed data. However, in real-world scenarios, numerous unpredictable factors give rise to the Out-of-Distribution (OOD) problem. Moreover, this distribution discrepancy may lead to unreliable predictions by GNNs models within unknown OOD domains. Therefore, this paper proposes a machine learning method to enhance the stability of GNN cross-site classification from the perspective of stable learning. The aim is to extract genuine causal features while eliminating spurious causal features. By introducing a feature sample weighting decorrelation technique in the random Fourier transform space and combining it with a baseline GNN model, a Stable-GNN model and a constrained sampling weight gradient update algorithm are designed. Its theoretical proof indicates that this algorithm can ensure the decrease in the loss, thus remedying the shortcomings of the original algorithm in weight updating. While guaranteeing predictive performance on the training distribution data, it also reduces prediction bias on data from unseen test distributions. Compared with the GNN model and the Stable-GNN model (S-GNN), the experimental results of the S-GNN model demonstrate that it not only surpasses the current state-of-the-art GNN models, but also offers a flexible framework for strengthening existing GNNs.

Graph Neural Networks (GNNs) is an important branch of deep learning in graph structure. As a model that can reveal deep topological information, GNNs has been widely used in various learning tasks, including physical system, protein interface prediction, disease classification, learning molecular fingerprints and so on. However, in most of these tasks, the graph data we are working with may be noisy and may contain spurious edges. That is to say, there is a lot of uncertainty associated with the underlying graph structure. The method of modeling uncertainty is to use Bayesian framework, in which graph is regarded as random variable. Introducing Bayesian framework into graph-based model, especially for semi-supervised node classification, has been shown that it can produce higher classification accuracy. In this paper, some GNNs models and Bayesian neural networks are introduced to better understand how GNNs are combined with Bayesian. Then, the development of Bayesian graph neural networks(BGNNs) in recent years is summarized, and its application in the field of engineering technology is demonstrated. Finally, the future development of BGNNs is prospected and the full text is summarized.

Drug response prediction is an essential part of precision medicine, which assesses the efficacy of drugs in specific cancer cell lines. Recently, numerous deep learning models have been developed for drug response prediction. The ability to effectively represent complicated interactions within a network makes Graph Neural Network (GNN) models a prominent area of exploration. To exploit the diverse capabilities of different GNN models, the proposed model uses an ensemble approach that combines results from Graph Convolutional Network (GCN), Graph SAmple and aggreGatE Network (GraphSAGE) and Graph Attention Network (GAT)-based models for drug response prediction. The proposed ensemble GNN model is used to predict the efficacy of drugs in treating leukemia. The model outperformed individual models when evaluated with the GDSC1000 dataset, with lower root mean squared error, higher Pearson’s correlation coefficient, higher coefficient of determination and higher Spearman’s correlation coefficient. Furthermore, most of the drugs identified by the model as potential candidates for the treatment of leukemia are currently in clinical trials or have supporting evidence in the PubMed literature database.

PurposeDrug-induced liver injury (DILI) is a significant concern in drug development, often leading to the discontinuation of clinical trials and the withdrawal of drugs from the market. This study explores the application of graph neural networks (GNNs) for DILI prediction, using molecular graph representations as the primary input.MethodsWe evaluated several GNN architectures, including Graph Convolutional Networks (GCNs), Graph Attention Networks (GATs), Graph Sample and Aggregation (GraphSAGE), and Graph Isomorphism Networks (GINs), using the latest FDA DILI dataset and other molecular property prediction datasets. We introduce a novel approach that creates a custom graph dataset, driven by molecular optimisation, that incorporates detailed and realistic chemical features such as bond lengths and partial charges as input into the GNN models. We have named our model approach DILIGeNN.ResultsDILIGeNN achieved an AUC of 0.897 on the DILI dataset, surpassing the current state-of-the-art model in the DILI prediction task. Furthermore, DILIGeNN outperformed the state-of-the-art in other graph-based molecular prediction tasks, achieving an AUC of 0.918 on the Clintox dataset, 0.993 on the BBBP dataset, and 0.953 on the BACE dataset, indicating strong generalisation and performance across different datasets.ConclusionDILIGeNN, utilising a single graph representation as input, outperforms the state-of-the-art methods in DILI prediction that incorporate both molecular fingerprint and graph-structured data. These findings highlight the effectiveness of our molecular graph generation and the GNN training approach as a powerful tool for early-stage drug development and drug repurposing pipeline.Scientific Contribution: DILIGeNN is a GNN framework that extracts graph features from 3D optimised molecular structures as is done in target-based drug discovery and molecular docking simulation. Our method is the first to encode spatial and electrostatic information into a single graph representation, as opposed to other work that require multiple graphs or additional chemical descriptors for feature representation. Our approach, using warm starts following repeated early stopping during training, outperforms the current state-of-the-art methods in liver toxicity (DILI), permeability (BBBP) and activity (BACE) prediction tasks.Graphic