Graph neural networks for integrated information and major complex estimation

We present an idea of protein molecules that challenges the traditional view of proteins as simple molecular machines and suggests instead that they exhibit a basic form of “intelligence”. The idea stems from suggestions coming from Integrated Information Theory (IIT), network theory, and allostery to explore how proteins process information, adapt to their environment, and even show memory-like behaviors. We define protein intelligence using IIT and focus on how proteins integrate information (in terms of the parameter Φ coming from IIT) and balance their core (stable, ordered regions) and periphery (flexible, disordered regions). This balance allows proteins to remain stable while adapting to changes and operating in a critical state where order and disorder coexist. We summarize recent findings on conformational memory, allosteric regulation, protein intrinsic disorder, liquid-liquid phase separation, and critical transitions, and compare protein behavior to other complex systems like ecosystems and neural networks. While our perspective offers a unified framework to understand proteins, it also raises questions about applying intelligence concepts to molecular systems. We discuss how this understanding could advance protein engineering, drug design, and synthetic biology, while at the same time acknowledging the challenges of creating adaptive, “intelligent” proteins. This concept bridges the gap between mechanistic and systems-level views of proteins and offers a comprehensive understanding of their dynamic and adaptive nature. We have tried to redefine the traditionally metaphorical concept of “intelligence” in biochemistry as a measurable property while simultaneously establishing the material foundation of protein intelligence through the identification of fundamental elements such as memory and learning in molecular systems.

A multi-agent artificial intelligence (AI) model is developed to automate the discovery of new metallic alloys, integrating multimodal data and external knowledge, including insights from physics via atomistic simulations. The system consists of (a) large language models (LLMs) for tasks such as reasoning and planning, (b) AI agents with distinct roles collaborating dynamically, and (c) a newly developed graph neural network (GNN) model for rapid retrieval of physical properties. We chose the ternary NbMoTa body-centered-cubic alloy as our model system and developed the GNN to predict two fundamental materials properties: the Peierls barrier and the solute/screw dislocation interaction energy. Our GNN model efficiently predicts these properties, reducing reliance on costly brute-force calculations and alleviating the computational demands on the multi-agent system. By combining the predictive capabilities of GNNs with the collaborative intelligence of LLM-driven reasoning agents, the system autonomously explores vast alloy design spaces, identifies trends in atomic-scale properties, and predicts macroscale mechanical strength, as demonstrated by several computational experiments. This synergistic approach accelerates the discovery of advanced alloys and holds promise for broader applications in other complex systems, marking a step forward in automated materials discovery and design. Impact statement Traditional deep learning models, such as graph neural networks and convolutional neural networks, operate within the confines of their training data sets, making single-step inferences for regression or classification. Our work introduces a multi-agent strategy that transcends these limitations by integrating deep learning with reasoning and decision-making capabilities. This intelligent system actively interprets results, determines subsequent actions, and iteratively refines predictions, accelerating the materials design process. We demonstrate its effectiveness in exploring the vast compositional space of a ternary alloy, where the model dynamically solicits data, analyzes trends, generates visualizations, and derives insights into materials behavior. By enabling accurate predictions of key alloy characteristics, our approach advances the discovery of novel metallic systems and underscores the critical role of solid-solution alloying. More broadly, it represents a major step toward integrating artificial intelligence with scientific reasoning, moving closer to artificial general intelligence in engineering. This paradigm shift has profound implications for materials science, enabling more efficient, autonomous, and intelligent exploration of complex materials spaces. Graphical Abstract

Recent advances in Graph Neural Networks (GNNs) have changed the landscape of modern graph analytics. The complexity of GNN training and the scalability challenges have also sparked interest from the systems community, with efforts to build systems that provide higher efficiency and schemes to reduce costs. However, we observe that many such systems basically "reinvent the wheel" of much work done in the database world on scalable graph analytics engines. Further, they often tightly couple the scalability treatments of graph data processing with that of GNN training, resulting in entangled complex problems and systems that often do not scale well on one of those axes. In this paper, we ask a fundamental question: How far can we push existing systems for scalable graph analytics and deep learning (DL) instead of building custom GNN systems? Are compromises inevitable on scalability and/or runtimes? We propose Lotan, the first scalable and optimized data system for full-batch GNN training with decoupled scaling that bridges the hitherto siloed worlds of graph analytics systems and DL systems. Lotan offers a series of technical innovations, including re-imagining GNN training as query plan-like dataflows, execution plan rewriting, optimized data movement between systems, a GNN-centric graph partitioning scheme, and the first known GNN model batching scheme. We prototyped Lotan on top of GraphX and PyTorch. An empirical evaluation using several real-world benchmark GNN workloads reveals a promising nuanced picture: Lotan significantly surpasses the scalability of state-of-the-art custom GNN systems, while often matching or being only slightly behind on time-to-accuracy metrics in some cases. We also show the impact of our system optimizations. Overall, our work shows that the GNN world can indeed benefit from building on top of scalable graph analytics engines. Lotan's new level of scalability can also empower new ML-oriented research on ever-larger graphs and GNNs.

Large cryogenic systems, like those installed at CERN, are complex systems relying on many diverse physical processes and phenomena that are difficult to simulate and monitor in detail. With only a limited number of properties measured and made available for monitoring and control purposes, several processes contributing to the dynamics of the systems are ignored. This lack of information can reduce the accuracy and the capability of a model to track, predict, and anticipate the behavior of the system. Accurate analytical or numerical computer modeling can be developed to simulate the non-linear dynamics of the processes but they are complex, computationally intensive, and cumbersome to test, validate, and implement with different configurations and limited measurements of the hidden properties. In this work, we present our investigation of using Graph Neural Networks (GNN) to build a model of the helium II bayonet heat exchanger operating in the LHC at CERN. We are proposing to use a hybrid machine learning approach, where the parameters of the GNN model are estimated by a combination of supervised learning algorithms trained on experimental data and bounding physics equations and parameters. The GNN model was initially trained on data from the experiments performed on the LHC prototype magnet strings and validated on data extracted during the operation of the LHC machine. We demonstrate the model’s accuracy, repeatability, and robustness in various configurations. The model is also well inspectable and explainable, providing the time evolution of all variables. We report on the results and expected applications, which include predictive control, diagnostic, and operator training.

Systems engineering is the branch of engineering concerned with the development of large and complex systems, where a system is understood to be an assembly or combination of interrelated elements or parts working together toward a common objective. Past experience has shown that formal systems engineering methodologies have not always been successfully applied to large and complex cybersecurity systems. These complex systems have become commonplace when applying cyberstrategies in cybersecurity operations. The ability to build, operate and maintain such systems is crucial to the effectiveness of cybersecurity operations. Most importantly, a cyberstrategy program must surround these systems on a global scale across multiple inter-related platforms. In this chapter, the authors demonstrate why a systems engineering approach is best suited for large and complex information systems used in cybersecurity, as well as the overall cyberstrategies that must also reside over these systems.

At present, the urban traffic system is faced with the problems of congestion and low efficiency, and the traditional methods have certain limitations when dealing with the complex urban road network. This paper aims to explore a new method of capacity Optimization of Road Networks in a Smart City environment and proposes a method based on a Graph Neural network, named “Smart City Road optimization Graph Neural Network” (SCRO-GNN). SCRO-GNN first collects and preprocesses multi-source data, including road network data, traffic flow, accident records, environmental factors, etc. The key node and edge features, including the number of lanes at the intersection, the traffic flow of the section, and the adjacency matrix of the road network are defined to characterize the structure of the road network. Then, the graph neural network model is constructed and trained to predict the traffic flow of different sections and evaluate the road capacity by using the graph structure of the road network. This paper tests the performance of SCRO-GNN on real road networks in multiple cities. The results show that, compared with the traditional traffic flow prediction model, SCRO-GNN can significantly improve the prediction accuracy, especially when dealing with the highly complex urban road network structure. Based on these predictions, the proposed optimization strategy performed well in reducing traffic congestion and improving the efficiency of road use. The research in this paper not only demonstrates the potential of graph neural networks in smart city road network optimization, but also provides a new direction for future traffic system research. The successful implementation of the SCRO-GNN approach is expected to provide more efficient and intelligent solutions for urban traffic management and planning.

Sex classification is a major benchmark of previous work in learning on the structural connectome, a naturally occurring brain graph that has proven useful for studying cognitive function and impairment. While graph neural networks (GNNs), specifically graph convolutional networks (GCNs), have gained popularity lately for their effectiveness in learning on graph data, achieving strong performance in adult sex classification tasks, their application to pediatric populations remains unexplored. We seek to characterize the capacity for GNN models to learn connectomic patterns on pediatric data through an exploration of training techniques and architectural design choices. Two datasets comprising an adult BRIGHT dataset (N = 147 Hodgkin's lymphoma survivors and N = 162 age similar controls) and a pediatric Human Connectome Project in Development (HCP-D) dataset (N = 135 healthy subjects) were utilized. Two GNN models (GCN simple and GCN residual), a deep neural network (multi-layer perceptron), and two standard machine learning models (random forest and support vector machine) were trained. Architecture exploration experiments were conducted to evaluate the impact of network depth, pooling techniques, and skip connections on the ability of GNN models to capture connectomic patterns. Models were assessed across a range of metrics including accuracy, AUC score, and adversarial robustness. GNNs outperformed other models across both populations. Notably, adult GNN models achieved 85.1% accuracy in sex classification on unseen adult participants, consistent with prior studies. The extension of the adult models to the pediatric dataset and training on the smaller pediatric dataset were sub-optimal in their performance. Using adult data to augment pediatric models, the best GNN achieved comparable accuracy across unseen pediatric (83.0%) and adult (81.3%) participants. Adversarial sensitivity experiments showed that the simple GCN remained the most robust to perturbations, followed by the multi-layer perceptron and the residual GCN. These findings underscore the potential of GNNs in advancing our understanding of sex-specific neurological development and disorders and highlight the importance of data augmentation in overcoming challenges associated with small pediatric datasets. Further, they highlight relevant tradeoffs in the design landscape of connectomic GNNs. For example, while the simpler GNN model tested exhibits marginally worse accuracy and AUC scores in comparison to the more complex residual GNN, it demonstrates a higher degree of adversarial robustness.

Machine learned potentials based on artificial neural networks are becoming a popular tool to define an effective energy model for complex systems, either incorporating electronic structure effects at the atomistic resolution, or effectively renormalizing part of the atomistic degrees of freedom at a coarse-grained resolution. One main criticism regarding neural network potentials is that their inferred energy is less interpretable than in traditional approaches, which use simpler and more transparent functional forms. Here we address this problem by extending tools recently proposed in the nascent field of explainable artificial intelligence to coarse-grained potentials based on graph neural networks. With these tools, neural network potentials can be practically decomposed into n-body interactions, providing a human understandable interpretation without compromising predictive power. We demonstrate the approach on three different coarse-grained systems including two fluids (methane and water) and the protein NTL9. The obtained interpretations suggest that well-trained neural network potentials learn physical interactions, which are consistent with fundamental principles.

Graph neural networks (GNNs) have made rapid developments in the recent years. Due to their great ability in modeling graph-structured data, GNNs are vastly used in various applications, including high-stakes scenarios such as financial analysis, traffic predictions, and drug discovery. Despite their great potential in benefiting humans in the real world, recent study shows that GNNs can leak private information, are vulnerable to adversarial attacks, can inherit and magnify societal bias from training data and lack interpretability, which have risk of causing unintentional harm to the users and society. For example, existing works demonstrate that attackers can fool the GNNs to give the outcome they desire with unnoticeable perturbation on training graph. GNNs trained on social networks may embed the discrimination in their decision process, strengthening the undesirable societal bias. Consequently, trust-worthy GNNs in various aspects are emerging to prevent the harm from GNN models and increase the users’ trust in GNNs. In this paper, we give a comprehensive survey of GNNs in the computational aspects of privacy, robustness, fairness, and explainability. For each aspect, we give the taxonomy of the related methods and formulate the general frameworks for the multiple categories of trustworthy GNNs. We also discuss the future research directions of each aspect and connections between these aspects to help achieve trustworthiness.

Mass and energy rate (ER) data have been collected for a wide variety of (complex) systems from the biological, cultural, and cosmological realms. They range from the cytochrome oxidase protein (10-22 kg and 6x10-19 W) to the observable universe (1.5x1053 kg and 1048 W) and, thus, span 75 mass and 66 ER orders of magnitude. Many of these systems are relevant for the big history (BH) narrative, i.e., the development of complexity over “big time” from the Big Bang up to the human society on Earth of today. The purpose of this paper is not per se to describe their history though, but to explore a master plot of ER vs. mass. Notably, the development of systems over big time has followed a rather tortuous path criss-crossing over this ER vs. mass master plot. The true mass of the system as a whole is used (for example trees including the non-living wood, living organisms including their intrinsic water, and social systems including the built constructs), because these inactive parts are essential for the performance of the system and facilitate its ER. A double logarithmic master plot of all ER vs. mass data shows clusters of data points. To some extent, this provides quantitative support for the distinction between the (sub-)realms, which is based on a qualitative description of their material structure and energy processing. In the master plot, small systems with low mass and ER converge into larger systems with larger mass and ER, which is typically accompanied by a decrease of the energy rate density (ERD = ER/mass). Correlation of ER with mass for various groups of systems demonstrates both sub- and supra-linear scaling with the power law β constant varying between 0.5 and 4.0, showing that the mechanisms of self-organisation are quite different for the corresponding system groups. The combination of convergence and scaling with β always larger than zero explains why the ER & mass data points fall in a diagonal band with a width of 17 orders of magnitude.
 ER and mass have changed over wide ranges during the evolution of groups of systems, suggesting that evolution can be viewed as a process of systems exploring a larger ER vs. mass area until they run into ER and/or mass limitations. Indeed, there is a diagonal ER vs. mass limit for stable systems in all realms, corresponding to an ERD value of around 105 W/kg. Systems with ER & mass combinations above this limit, such as bombs, super-novae and cosmological transients, are unstable and “explosive”. This raises the interesting question of whether such an ERD maximum puts a limit on the development of complexity over big time. It seems that the low, right side of the master plot is empty. However, it is argued here that it is full of systems with low ER, such as dormant, living organisms, technological systems with their power adjusted or even switched off, as well as cooling, cosmological objects. Such systems are typically considered of less interest in a BH context, but they are viewed here as simple, complex systems which are out of equilibrium with matter, energy and information stored in their structure. While ERD appears to increase with the ‘advancement’ of systems over big time [5,51,52], there are quite some confounding factors regarding the efficacy of ERD as a metric for complexity in BH. For example, ERD decreases during the lifetime of a human and the human society (the mass of human-made constructs has grown faster than the global energy consumption), as well as during the evolution of living organisms and stars, whereas complexity is considered to increase. High ERD values of system parts may be illustrative for the complexity of the larger system, but are not representative for ERD of the system itself. Machines with an increased efficiency of energy conversion have a lower ERD, but could be considered more complex. The smallest and largest ERD values observed for the various realms appear to correlate with activity level and reciprocally with size, which do not per se reflect complexity. It is hoped that the raw data collected and the major trends observed in this paper will offer new insights into various aspects of the evolution of the universe over big time, and serve as an important resource for other related studies.

Past events have revealed that widespread blackouts are mostly a result of cascading failures in the power grid. Understanding the underlining mechanisms of cascading failures can help in developing strategies to minimize the risk of such events. Moreover, a real-time detection of precursors to cascading failures will help operators take measures to prevent their propagation. Currently, the well-established probabilistic and physics-based models of cascading failures offer low computational efficiency, hindering them to be used only as offline tools. In this work, we develop a data-driven methodology for online estimation of the risk of cascading failures. We utilize a physics-based cascading failure model to generate a cascading failure dataset considering different operating conditions and failure scenarios, thus obtaining a sample space covering a large set of power grid states that are labeled as safe or unsafe. We use the synthetic data to train deep learning architectures, namely Feed-forward Neural Networks (FNN) and Graph Neural Networks (GNN). With the development of GNNs, improved performance is achieved with graph-structured data, and GNNs can generalize to graphs of diverse sizes. A comparison between FNN and GNN is made and the GNNs inductive capability is demonstrated via test grids. Furthermore, we apply transfer learning to improve the performance of a pre-trained GNN model on power grids not seen in the training process. The GNN model shows accuracy and balanced accuracy above 96% on selected test datasets not used in the training. Conversely, the FNN shows accuracy above 85% and balanced accuracy above 81% on test datasets unseen during training. Overall, the GNN model is successful in determining, if one or several simultaneous outages result in a critical grid state, under specific grid operating conditions.

The n-octanol/buffer solution distribution coefficient at pH = 7.4 (log D7.4) is an indicator of lipophilicity, and it influences a wide variety of absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties and druggability of compounds. In log D7.4 prediction, graph neural networks (GNNs) can uncover subtle structure-property relationships (SPRs) by automatically extracting features from molecular graphs that facilitate the learning of SPRs, but their performances are often limited by the small size of available datasets. Herein, we present a transfer learning strategy called pretraining on computational data and then fine-tuning on experimental data (PCFE) to fully exploit the predictive potential of GNNs. PCFE works by pretraining a GNN model on 1.71 million computational log D data (low-fidelity data) and then fine-tuning it on 19,155 experimental log D7.4 data (high-fidelity data). The experiments for three GNN architectures (graph convolutional network (GCN), graph attention network (GAT), and Attentive FP) demonstrated the effectiveness of PCFE in improving GNNs for log D7.4 predictions. Moreover, the optimal PCFE-trained GNN model (cx-Attentive FP, Rtest2 = 0.909) outperformed four excellent descriptor-based models (random forest (RF), gradient boosting (GB), support vector machine (SVM), and extreme gradient boosting (XGBoost)). The robustness of the cx-Attentive FP model was also confirmed by evaluating the models with different training data sizes and dataset splitting strategies. Therefore, we developed a webserver and defined the applicability domain for this model. The webserver (http://tools.scbdd.com/chemlogd/) provides free log D7.4 prediction services. In addition, the important descriptors for log D7.4 were detected by the Shapley additive explanations (SHAP) method, and the most relevant substructures of log D7.4 were identified by the attention mechanism. Finally, the matched molecular pair analysis (MMPA) was performed to summarize the contributions of common chemical substituents to log D7.4, including a variety of hydrocarbon groups, halogen groups, heteroatoms, and polar groups. In conclusion, we believe that the cx-Attentive FP model can serve as a reliable tool to predict log D7.4 and hope that pretraining on low-fidelity data can help GNNs make accurate predictions of other endpoints in drug discovery.

Network modeling is a critical component for building self-driving Software-Defined Networks, particularly to find optimal routing schemes that meet the goals set by administrators. However, existing modeling techniques do not meet the requirements to provide accurate estimations of relevant performance metrics such as delay and jitter. In this paper we propose a novel Graph Neural Network (GNN) model able to understand the complex relationship between topology, routing and input traffic to produce accurate estimates of the per-source/destination pair mean delay and jitter. GNN are tailored to learn and model information structured as graphs and as a result, our model is able to generalize over arbitrary topologies, routing schemes and variable traffic intensity. In the paper we show that our model provides accurate estimates of delay and jitter (worst case R2 = 0.86) when testing against topologies, routing and traffic not seen during training. In addition, we present the potential of the model for network operation by presenting several use-cases that show its effective use in per-source/destination pair delay/jitter routing optimization and its generalization capabilities by reasoning in topologies and routing schemes not seen during training.

Neural relational and dynamics inference for complex systems

Graph Neural Networks (GNNs) have emerged as a powerful framework for processing irregular, complex systems such as social networks and knowledge graphs. However, the inherent structural complexity of graphs imposes significant memory and computational overhead. These challenges are exacerbated by the traditional von Neumann architecture, where physically separated memory and processing units lead to substantial data movement bottlenecks. Consequently, in-memory computing (IMC) paradigms—specifically those leveraging memristor crossbar arrays—have become increasingly vital. While extensive research has focused on utilizing memristor arrays as weight layers for neural network acceleration, the adjacency matrix—a fundamental component representing graph topology—remains largely unimplemented in hardware. In this work, we provide the first experimental demonstration of a GNN architecture where the adjacency matrix is physically realized using a memristor crossbar array, harnessing its massive parallel computing capabilities. By integrating this hardware-based adjacency matrix with software-defined layers, we successfully implement digital image processing and resolve complex combinatorial optimization problems. Our results exhibit significantly enhanced computational efficiency and underscore the potential of memristor arrays as core components in GNN acceleration. This physical implementation of the adjacency matrix expands the horizons of memristor-based neuromorphic computing systems, offering a scalable path toward high-performance machine learning and artificial intelligence hardware.