Survey on Graph-Based Reinforcement Learning for Networked Coordination and Control

This paper presents a preliminary study on bulk power system reliability assessment using graph neural networks. The proposed method is an end-to-end pipeline that can directly predict the reliability index. The Monte Carlo simulation and the end-to-end machine learning paradigm for bulk power system reliability assessment are introduced. Then, the basic principles of graph signal processing and graph neural networks are explained. Dataset generation and feature engineering for applying graph neural networks on end-to-end reliability assessment under situations of system topology-change are illustrated in detail. Experiment results on the RTS-79 system by the proposed graph neural networks pipeline demonstrate an obvious speed improvement over the regular Monte Carlo simulation method with acceptable prediction errors. Future research directions are also suggested in the final section.

This paper investigates the routing problem in the mega low earth orbit (mLEO) satellite constellations considering factors including distribution of the users, topology of the networks and dynamics of the inter-satellite links (ISLs). In walker-delta constellations, each satellite establishes four stable ISLs with neighboring satellites to forward data packets without relying on ground facilities. In order to minimize the delay from the source satellite to the destination satellite, the routing problem is formulated as a Markov decision problem (MDP) and the GRLR routing algorithm is proposed which integrates reinforcement learning (RL) and graph neural network (GNN) effectively. The GRLR establishes the decision network based on Actor-Critic RL framework and builds feature extraction network based on GNN to realize distributed intelligent routing decisions in mLEO constellations. Finally, the simulation experiments are carried out to illustrate that the proposed strategy exhibits rapid convergence and outperforms the baseline strategies in terms of delay and adaptability to network dynamics.

In recent years, with the rapid development of network applications and the increasing demand for high-quality network service, quality-of-service (QoS) routing has emerged as a critical network technology. The application of machine learning techniques, particularly reinforcement learning and graph neural network, has garnered significant attention in addressing this problem. However, existing reinforcement learning methods lack research on the causal impact of agent actions on the interactive environment, and graph neural network fail to effectively represent link features, which are pivotal for routing optimization. Therefore, this study quantifies the causal influence between the intelligent agent and the interactive environment based on causal inference techniques, aiming to guide the intelligent agent in improving the efficiency of exploring the action space. Simultaneously, graph neural network is employed to embed node and link features, and a reward function is designed that comprehensively considers network performance metrics and causality relevance. A centralized reinforcement learning method is proposed to effectively achieve QoS-aware routing in Software-Defined Networking (SDN). Finally, experiments are conducted in a network simulation environment, and metrics such as packet loss, delay, and throughput all outperform the baseline.

Traffic Engineering (TE) is an important network operation that routes/reroutes flows based on network topology and traffic demands to optimize network performance. Recently, new emerging applications pose challenges to TE with dynamic network conditions, where frequent routing updates are required to maintain good network performance with Software-Defined Networking (SDN). However, flow rerouting operations could lead to considerable Quality of Service (QoS) degradation and service disruption, which is often neglected by existing TE solutions. In this paper, we apply a new QoS metric named network disturbance to measure the negative impact of flow rerouting operations performed by TE. To achieve near-optimal load balancing performance and mitigate network disturbance together in dynamic network scenarios, we propose a flexible and disturbance-aware TE solution called FlexDATE that combines Reinforcement Learning (RL) and Linear Programming (LP). Specifically, FlexDATE leverages RL to intelligently identify flexible numbers of critical flows for each traffic matrix and reroutes these critical flows based on LP optimization to improve network performance with low disturbance. Empowered by a customized actor-critic architecture coupled with Graph Neural Networks (GNNs), FlexDATE can generalize well to unseen traffic scenarios and remain resilient to single link failures. Extensive simulations are conducted on five real-world network topologies to evaluate FlexDATE with real and synthetic traffic traces. The results show that FlexDATE can achieve the performance target (i.e., 90% of optimal performance) in 99% of network scenarios and effectively mitigate the average and maximum network disturbance by up to 9.1% and 38.6%, respectively, compared to state-of-the-art TE solutions.

Graph neural networks (GNNs) have made rapid developments in the recent years. Due to their great ability in modeling graph-structured data, GNNs are vastly used in various applications, including high-stakes scenarios such as financial analysis, traffic predictions, and drug discovery. Despite their great potential in benefiting humans in the real world, recent study shows that GNNs can leak private information, are vulnerable to adversarial attacks, can inherit and magnify societal bias from training data and lack interpretability, which have risk of causing unintentional harm to the users and society. For example, existing works demonstrate that attackers can fool the GNNs to give the outcome they desire with unnoticeable perturbation on training graph. GNNs trained on social networks may embed the discrimination in their decision process, strengthening the undesirable societal bias. Consequently, trust-worthy GNNs in various aspects are emerging to prevent the harm from GNN models and increase the users’ trust in GNNs. In this paper, we give a comprehensive survey of GNNs in the computational aspects of privacy, robustness, fairness, and explainability. For each aspect, we give the taxonomy of the related methods and formulate the general frameworks for the multiple categories of trustworthy GNNs. We also discuss the future research directions of each aspect and connections between these aspects to help achieve trustworthiness.

Recommendation systems is the core of most online experiences, guiding people to the right products, films, music, courses, and even social links when the choices are overwhelming. They work by learning from large interaction logs and applying predictive models to surface content that fits each user’s tastes, steadily improving as algorithms and data grow richer. As AI and deep learning have developed, classic strategies like collaborative filtering and content-based filtering are increasingly paired with, or superseded by, neural architectures that can model more nuanced patterns in behaviour and content. Among these neural approaches, graph neural networks—and especially Graph Attention Networks (GATs)—have become influential for recommendation because they treat users and items as nodes in a graph and learn from the connections between them. With attention mechanisms, GATs decide which neighbours matter most for a given prediction, amplifying useful signals and dampening noise to learn stronger user and item representations. This survey traces how the industry moved from traditional methods to graph-based models centred on GATs, comparing approaches on scalability, ranking accuracy, data sparsity, and interpretability across widely used benchmarks. Drawing from recent peer‑reviewed work, it highlights architectures and training practices that consistently help in real-world settings, as well as gaps that still limit robust deployment at scale. Looking forward, several directions stand out: using explainable AI to clarify why specific items are recommended, combining reinforcement learning with GATs for adaptive policies, and adopting federated learning to protect user privacy while training on distributed data. The versatility of GAT-based recommenders is already visible across domains—from e‑commerce and online learning to healthcare and social feeds—where graph structure and selective attention naturally fit the data. Overall, the evidence points to GATs as a key ingredient for the next generation of personalized, context‑aware, and trustworthy recommendation systems. Keywords: Recommendation Systems, Collaborative Filtering, Graph Neural Networks, Graph Attention Networks

Autonomous mobility-on-demand (AMoD) systems represent a rapidly developing mode of transportation wherein travel requests are dynamically handled by a coordinated fleet of robotic, self-driving vehicles. Given a graph representation of the transportation network - one where, for example, nodes represent areas of the city, and edges the connectivity between them - we argue that the AMoD control problem is naturally cast as a node-wise decision-making problem. In this paper, we propose a deep reinforcement learning framework to control the rebalancing of AMoD systems through graph neural networks. Crucially, we demonstrate that graph neural networks enable reinforcement learning agents to recover behavior policies that are significantly more transferable, generalizable, and scalable than policies learned through other approaches. Empirically, we show how the learned policies exhibit promising zero-shot transfer capabilities when faced with critical portability tasks such as inter-city generalization, service area expansion, and adaptation to potentially complex urban topologies.

In mountainous environments, terrain occlusion causes non-line-of-sight (NLoS) transmission, significantly reducing the signal propagation range. To improve emergency rescue efficiency, a mobile unmanned aerial vehicle (UAV)-based integrated localization and communication (ILAC) network should be deployed to achieve optimal performance through adaptive trajectory planning and resource allocation. However, irregular and unpredictable terrain occlusions, coupled with dynamic users, make traditional optimization ineffective and reinforcement learning (RL) inefficient. To address these challenges, this paper proposes a hybrid action space soft Actor-Critic with geographic information-based state aggregation (GeoAgg-HSAC) decision-making scheme. First, an RL state aggregation method based on graph contrastive learning is designed. Through a pre-trained graph neural network (GNN), the UAV network states experiencing the same occlusion are mapped to similar low-dimensional representations. This method reduces the state dimension and allows similar states to share policy experience, thereby improving sample efficiency and accelerating convergence. A hybrid action space SAC network is then designed, which simultaneously makes decisions for continuous UAV trajectories and discrete resource allocation. Finally, a simulation environment based on real mountain terrain and wireless data is built for the experiment. The experimental results show that the proposed scheme has significant advantages for optimizing communication and localization performance.

In general, graph neural networks (GNNs) adopt the message-passing scheme to capture the information of a node (i.e., nodal attributes, and local graph structure) by iteratively transforming, aggregating the features of its neighbors. Nonetheless, recent studies show that the performance of GNNs can be easily hampered by the existence of abnormal or malicious nodes due to the vulnerability of neighborhood aggregation. Thus it is necessary to learn anomaly-resistant GNNs without the prior knowledge of ground-truth anomalies, given the fact that labeling anomalies is costly and requires intensive domain knowledge. Though removing anomalies through unsupervised anomaly detection methods could be a possible solution, it may render unreasonable GNN model performance on target tasks due to the non-differentiable gap between the two learning procedures. In order to keep the effectiveness of GNNs on anomaly-contaminated graphs, in this paper, we propose a new framework named RARE-GNN (Reinforced Anomaly-REsistant Graph Neural Networks) which can detect anomalies from the input graph and learn anomaly-resistant GNNs simultaneously. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed framework.

This paper surveys the growing interest in utilizing deep learning (DL) as a powerful tool to address challenging problems in earthquake engineering. Despite decades of advancement in domain knowledge, issues such as uncertainty in earthquake occurrence, unpredictable seismic loads, nonlinear structural responses, and community engagement remain difficult to tackle using domain-specific methods. DL offers promising solutions by leveraging its data-driven capacity for nonlinear mapping, sequential data modeling, automatic feature extraction, dimensionality reduction, optimal decision-making, and so on. However, the literature lacks a comprehensive review that systematically covers a consistent scope intersecting DL and earthquake engineering. To bridge this gap, this paper first discusses methodological advances to elucidate various applicable DL techniques, such as multilayer perceptron, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network, autoencoder, transfer learning, reinforcement learning (RL), and graph neural network. A thorough research landscape is then disclosed by exploring various DL applications across different research topics, including vision-based seismic damage assessment and structural characterization, seismic demand and damage state prediction, seismic response history prediction, regional seismic risk assessment and community resilience, ground motion (GM) for engineering use, seismic response control, and the inverse problem of system/damage identification. Suitable DL techniques for each research topic are identified, emphasizing the preeminence of CNN for vision-based tasks, RNN for sequential data, RL for community resilience, and unsupervised learning for GM analysis. The paper also discusses opportunities and challenges for leveraging DL in earthquake engineering research and practice, highlighting the need for open-access multimodal big data and efforts to enhance model interpretability and incorporate physics information into DL. Finally, the paper advocates for DL applications to further advance the research frontier of uncertainty quantification in performance-based earthquake engineering.

Network dismantling is one of the most challenging problems in complex systems. This problem encompasses a broad array of practical applications. Previous works mainly focus on the metrics such as the number of nodes in the Giant Connected Component (GCC), average pairwise connectivity, etc. This paper introduces a novel metric, the accumulated 2-core size, for assessing network dismantling. Due to the NP-hard computational complexity of this problem, we propose SmartCore, an end-to-end model for minimizing the accumulated 2-core size by leveraging reinforcement learning and graph neural networks. Extensive experiments across synthetic and real-world datasets demonstrate SmartCore’s superiority over existing methods in terms of both accuracy and speed, suggesting that SmartCore should be a better choice for the network dismantling problem in practice.

Autonomous driving and artificial intelligence are the most popular research projects in the field of technology today. As the high technology, autonomous driving relies on perception, decision-making, and control systems, and the performance of these systems largely depends on the application of artificial intelligence nowadays. Fortunately, there are plenty of applications of artificial intelligence in several aspects of autonomous driving. This paper aims to introduce the relationship between autonomous driving and artificial intelligence by reviewing several literatures and analyzing the applications of deep learning (DL), reinforcement learning (RL), and graph neural networks (GNN) in autonomous driving.

The cities are the largest emitters of the greenhouse gas (GHG) emissions in the world, with close to a quarter of the total energy‐related CO 2 emissions occurring in urban places. As urbanization and demand to move surge in transiting, traditional monomodal transport systems, especially the ones that rely on the use of personal vehicles, are inefficient given the low efficiency, and thus unaffordable to support the goal of sustainability. The current article suggests the idea of an artificial intelligence (AI)‐powered, Internet of Things (IoT)‐compatible real‐time multimodal optimization tool to minimize urban emissions yet improve the efficient mobility of people and goods. The system unites AI algorithms, such as reinforcement learning (RL) and graph neural networks (GNNs), and the data streams based on the IoT devices connected to road sensors, GPS, mobile apps, and the public transportation systems. It actively suggests the best paths and combinations of methods of transport (e.g., walking, biking, bus, metro) according to the traffic, weather, user preferences, and emission characteristics in real time. Through simulations that have been carried out in the simulation of urban mobility (SUMO) platform on a modeled city, average CO 2 emissions per trip have been reduced by 45% compared with baseline routing strategies. Further findings indicate the use of suggested alternatives with high user acceptance rate (83%) and efficient modal shifts of the avoided use of a private car. The suggested framework provides a data‐driven and scalable framework of sustainable urban mobility. Considering that it integrates modeling of emissions into the central optimization engine, enables real‐time, personalized decision‐making, it can help achieve climate action as well as aid in the smart city development. The proposed AI–IoT framework demonstrably reduces CO 2 emissions and supports the United Nations Sustainable Development Goals (SDGs), specifically SDG 11 (cities, targets 11.2/11.6), SDG 13 (climate, target 13.2), and SDG 9 (infrastructure, targets 9.1/9.4) with secondary alignment to SDG 7.3 on energy efficiency.

The burgeoning changes involved in 5G networks bring forth unseen opportunities and challenges in the provision and support of bright, scalable and adaptable services across a wide range of applications. The richness of heterogeneity, time-varying service needs, and the demanding one-order latency necessitate new solutions that far exceed traditional network slicing. The proposed unified intelligent framework of intelligent network slicing, i.e., Intelligence Slicing, is dedicated to cross-domain resource orchestration and service optimization. The framework incorporates the use of Artificial Intelligence (AI) at the center of network slicing mechanisms that enable the autonomous management and optimization of slices, in the domains comprising the RAN, the transport network, the core network, and cloud/edge computing. We study methods such as Reinforcement Learning (RL), Federated Learning (FL), and Graph Neural Networks (GNN) for learning decentralized intelligence, and design them so that optimizing them in real time to traffic requirements, user mobility and Service Level Agreement (SLA). The parameters of the proposed model are simulated under various 5G traffic conditions, including URLLC, eMBB, and mMTC. Performance streams are used with slice utilization, latency, throughput and energy efficiency being analyzed. The findings show that there was a 35 percent increase in resource utilization and a 27 percent decrease in the end-to-end latency ratio compared to the traditional methods that are heuristic-based. This will provide the foundations of what an AI-native 6G network will be.

Synthetically analyze the power distribution communication network technology in the smart power grid link. It mainly includes the summary of the smart distribution network, electric power communication backbone network and then layer communication requirements, the network mode and the application mode of WLAN technology in the distribution network, EPON network technology, micropower wireless technology, PTN technology and power broadband wireless systems. State the demand of the smart power distribution business and put forward technical guidance for the smart power distribution communication network scheme. In view of the complex distribution network architecture, more nodes and scattered, complex business types data business needs, put forward the integrated solutions of optical fiber communication as the backbone access, wireless bandwidth, passive optical communication as the coverage, the short distance wireless communication as the main terminal access. It provides a reference for the communication plan of distribution side.