A Hierarchical Reinforcement Learning Framework for Multi-Agent Cooperative Maneuver Interception in Dynamic Environments

The Dual Active Bridge converter (DABC), known for its bidirectional power transfer capability and high efficiency, plays a crucial role in various applications, particularly in electric vehicles (EVs), where it facilitates energy storage, battery charging, and grid integration. The Dual Active Bridge Converter (DABC), when paired with a high-performance CLLC filter, is well-regarded for its ability to transfer power bidirectionally with high efficiency, making it valuable across a range of energy applications. While these features make the DABC highly efficient, they also complicate controller design due to nonlinear behavior, fast switching, and sensitivity to component variations. We have used a Fractional-order PID (FOPID) controller to benefit from the simple structure of classical PID controllers with lower complexity and improved flexibility because of additional filtering gains adopted in this method. However, for a FOPID controller to operate effectively under real-time conditions, its parameters must adapt continuously to changes in the system. To achieve this adaptability, a Multi-Agent Reinforcement Learning (MARL) approach is adopted, where each gain of the controller is tuned individually using the Deep Deterministic Policy Gradient (DDPG) algorithm. This structure enhances the controller’s ability to respond to external disturbances with greater robustness and adaptability. Meanwhile, finding the best initial gains in the RL structure can decrease the overall efficiency and tracking performance of the controller. To overcome this issue, Grey Wolf Optimization (GWO) algorithm is proposed to identify the most suitable initial gains for each agent, providing faster adaptation and consistent performance during the training process. The complete approach is tested using a Hardware-in-the-Loop (HIL) platform, where results confirm accurate voltage control and resilient dynamic behavior under practical conditions. In addition, the controller’s performance was validated under a battery management scenario where the DAB converter interacts with a nonlinear lithium-ion battery. The controller successfully regulated the State of Charge (SOC) through automated charging and discharging transitions, demonstrating its real-time adaptability for BMS-integrated EV systems. Consequently, the proposed MARL-FOPID controller reported better disturbance-rejection performance in different working cases compared to other conventional methods.

In this article, we explore how to optimize task allocation for robot swarms in dynamic environments, emphasizing the necessity of formulating robust, flexible, and scalable strategies for robot cooperation. We introduce a novel framework using a decentralized partially observable Markov decision process (Dec-POMDP), specifically designed for distributed robot swarm networks. At the core of our methodology is the local information aggregation multiagent deep deterministic policy gradient (LIA-MADDPG) algorithm, which merges centralized training with distributed execution. During the centralized training phase, a local information aggregation (LIA) module is meticulously designed to gather critical data from neighboring robots, enhancing decision-making efficiency. In the distributed execution phase, a strategy improvement method is proposed to dynamically adjust task allocation based on changing and partially observable environmental conditions. Our empirical evaluations show that the LIA module can be seamlessly integrated into various centralized training and decentralized execution (CTDE)-based multiagent reinforcement learning (MARL) methods, significantly enhancing their performance. Additionally, by comparing LIA-MADDPG with six conventional reinforcement learning algorithms and a heuristic algorithm, we demonstrate its superior scalability, rapid adaptation to environmental changes, and ability to maintain both stability and convergence speed. These results underscore LIA-MADDPG's outstanding performance and its potential to significantly improve dynamic task allocation in robot swarms through enhanced local collaboration and adaptive strategy execution.

The large-scale integration of distributed renewable generation (DRG) reduces carbon emissions in power systems, but its intermittency and volatility introduce new challenges for voltage and frequency control. This paper proposes a multi-time-scale reactive power optimization approach that incorporates demand-side response. Firstly, based on the interaction characteristics between sources and loads in the distribution network, a demand-side response mechanism using dynamic time-of-use pricing is designed to explore the source-load matching and tracking capabilities. Secondly, a multi-scale day-ahead reactive power optimization control model is constructed, which comprehensively considers the dynamic response mechanism on the demand side. The objective function of this model aims to minimize network losses and voltage deviations while accounting for the day-ahead price response characteristics of loads. The multi-scale reactive power optimization model is then transformed into a Markov decision process and solved via a locally-observed multi-agent deep deterministic policy gradient algorithm to obtain real-time intraday reactive power and voltage control strategies. Finally, the effectiveness of the proposed method is verified through experiments on the IEEE 33-bus system and an actual power grid. Experimental results show that this method can effectively coordinate various reactive power compensation devices and flexible resources, and improve the efficiency and real-time performance of reactive power and voltage control.

PASCAL: PopulAtion-Specific Curriculum-based MADRL for collision-free flocking with large-scale fixed-wing UAV swarms

Vehicular networks face significant challenges in resource allocation due to heterogeneous Quality-of-Service (QoS) requirements and dynamic channel conditions. This paper proposes an efficient resource allocation framework for heterogeneous vehicular networks to address diverse QoS requirements in dynamic environments. We introduce a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm enhanced by Graph Attention Networks (GATs), named MADDPG-GAT, to optimize sub-bands allocation and power control for Vehicle-to-Vehicle (V2V) links, while minimizing interference to Vehicle-to-Infrastructure (V2I) links. By constructing a hierarchical relationship graph to capture intra-group and inter-group interactions among vehicles, the proposed MADDPG-GAT leverages local and global embeddings for cooperative decision-making. The framework employs centralized offline training for efficient policy learning and decentralized online execution for real-time decision-making with low overhead, with normalized reward function to enhance convergence and training stability. Simulation results demonstrate that our approach outperforms baseline methods in terms of system performance and robustness.

Misinformation in online social networks (OSNs) has been an ongoing problem, and it has been studied heavily over recent years. In this article, we use gamification to tackle misinformation propagation in OSNs. First, we construct a game based on the notion of cooperative games on graphs where the nodes of the social network are players. We use random regular networks and real networks in our simulations to show that the constructed game follows evolutionary dynamics and that the outcome of the game depends on the relation between the structural properties of the network and the benefit and cost variables defined in a cooperative game. Second, we create a game on the network level where the players control a set of nodes. We define agents whose goal is to maximize the total reward that we set up to be the number of nodes affected at the end of the game. We propose a deep reinforcement learning (RL) technique based on the multiagent deep deterministic policy gradient (MADDPG) algorithm. We test the proposed method along with well-known node selection algorithms and obtain promising results on different social networks.

Multi-agent deep reinforcement learning based demand response for discrete manufacturing systems energy management

Industrial edge computing in time-sensitive, heterogeneous environments faces significant challenges in delivering deterministic, low-latency, and fair task offloading under mixed-criticality traffic. Existing solutions struggle with unpredictable delays, resource contention, and the joint optimization of offloading and scheduling. In this work, we propose a task queue mapping mechanism-based architecture that incorporates formal mixed-deterministic traffic modeling under time-sensitive networking, enabling unified computation and network resource management. The joint task offloading and traffic scheduling problem is rigorously formulated as a two-stage Markov decision process with embedded fairness constraints. To address the strong coupling and computational complexity, we develop a fairness-aware deterministic offloading and resource scheduling (FA-DORS) framework, a hierarchical multiagent deep deterministic policy gradient algorithm, enhanced by a contribution-based local reward mechanism to ensure adaptive resource allocation and fairness. Extensive experimental results demonstrate that in the proposed architecture, FA-DORS achieves lower task latency, higher completion rates, improved resource utilization, and enhanced transmission determinism while ensuring fairness in multitask offloading compared to baseline methods. In particular, FA-DORS achieves an 8.89%–39.15% reduction in average end-to-end delay and up to 39.73% reduction in average jitter under high load, compared with the baseline method.

Clusters of unmanned aerial vehicles (UAVs) are often used to perform complex tasks. In such clusters, the reliability of the communication network connecting the UAVs is an essential factor in their collective efficiency. Due to the complex wireless environment, however, communication malfunctions within the cluster are likely during the flight of UAVs. In such cases, it is important to control the cluster and rebuild the connected network. The asymmetry of the cluster topology also increases the complexity of the control mechanisms. The traditional control methods based on cluster consistency often rely on the motion information of the neighboring UAVs. The motion information, however, may become unavailable because of the interrupted communications. UAV control algorithms based on deep reinforcement learning have achieved outstanding results in many fields. Here, we propose a cluster control method based on the Decomposed Multi-Agent Deep Deterministic Policy Gradient (DE-MADDPG) to rebuild a communication network for UAV clusters. The DE-MADDPG improves the framework of the traditional multi-agent deep deterministic policy gradient (MADDPG) algorithm by decomposing the reward function. We further introduce the reward reshaping function to facilitate the convergence of the algorithm in sparse reward environments. To address the instability of the state-space in the reinforcement learning framework, we also propose the notion of the virtual leader–follower model. Extensive simulations show that the success rate of the DE-MADDPG is higher than that of the MADDPG algorithm, confirming the effectiveness of the proposed method.

Federated deep reinforcement learning for task offloading and resource allocation in mobile edge computing-assisted vehicular networks

Non-orthogonal multiple access (NOMA) and reconfigurable intelligent surfaces (RISs) are recognized as key technologies for beyond 5G and 6G wireless communications. To address the high computational complexity and non-convex optimization challenges, this letter proposes an optimization framework based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The proposed framework jointly makes use of sensor grouping, power allocation, an RIS computation strategy, and phase shifts to minimize the remote state estimation (RSE) error. Simulation results demonstrate that the MADDPG algorithm, when applied in an RIS-assisted NOMA system, significantly reduces the RSE error.

Introduction To address the urgent need for enhanced grid flexibility in high-penetration distributed photovoltaic (PV) systems, this paper proposes a novel two-stage framework integrating multi-objective cluster partitioning and multi-agent reinforcement learning (MARL) for cooperative peak shaving. Methods First, a comprehensive multi-objective optimization model defines PV cluster partitioning using three domain-specific metrics: available peak shaving margin (APSM), peak shaving synergy (PSS), and node correlation degree (NCD). This approach replaces conventional clustering algorithms, ensuring partitions balance electrical compactness, regulation capacity, and operational coordination. Subsequently, each partitioned cluster acts as an autonomous agent in a hierarchical MARL framework. Leveraging the multi-agent deep deterministic policy gradient (MADDPG) algorithm, agents collaboratively optimize active/reactive power allocation among clusters while coordinating unit-level control within clusters. Results Validated on the IEEE-33 node system, the strategy achieves a 29.39% reduction in peak-to-valley difference and 54.44% improvement in power balance deviation, significantly smoothing net load curves and enhancing voltage stability. Discussion This work demonstrates how MARL-driven coordination unlocks the inherent flexibility of distributed PV clusters, providing a scalable solution for grid-edge resource participation in system-level peak shaving.

The convergence of mobile edge computing (MEC) and blockchain is transforming the current computing services in mobile networks, by offering task offloading solutions with security enhancement empowered by blockchain mining. Nevertheless, these important enabling technologies have been studied separately in most existing works. This article proposes a novel cooperative task offloading and block mining (TOBM) scheme for a blockchain-based MEC system where each edge device not only handles data tasks but also deals with block mining for improving the system utility. To address the latency issues caused by the blockchain operation in MEC, we develop a new Proof-of-Reputation consensus mechanism based on a lightweight block verification strategy. A multi-objective function is then formulated to maximize the system utility of the blockchain-based MEC system, by jointly optimizing offloading decision, channel selection, transmit power allocation, and computational resource allocation. We propose a novel distributed deep reinforcement learning-based approach by using a multi-agent deep deterministic policy gradient algorithm. We then develop a game-theoretic solution to model the offloading and mining competition among edge devices as a potential game, and prove the existence of a pure Nash equilibrium. Simulation results demonstrate the significant system utility improvements of our proposed scheme over baseline approaches.

Vehicle-to-vehicle communications can offer direct information interaction, including security-centered information and entertainment information. However, the rapid proliferation of vehicles and the diversity of communications services demand for a more intelligent and efficient resource allocation framework to enhance network performance. In this article, a multi-agent deep reinforcement learning-based resource allocation framework is developed to jointly optimize the channel allocation and power control to satisfy the heterogeneous Quality-of-Service (QoS) requirements in heterogeneous vehicular networks. In the proposed framework, the utility maximization problem is formulated by considering two types of traffics, i.e., the strict ultrareliable and low-latency requirements for safety-centric applications and the high-capacity requirements for entertainment applications. The utility of each vehicular users is formulated as a multicriterion objective function by taking into account the heterogeneous traffic requirements. To overcome the drawbacks of the traditional totally centralized and distributed deep reinforcement learning-based resource allocation approaches, we propose a multi-agent deep deterministic policy gradient algorithm with centralized learning and decentralized execution to solve the formulated optimization problem. The normalization of the input states and reward functions is introduced to speed up the training and learning progress of the proposed algorithm. Simulation results show the superiority of the proposed algorithm in terms of the convergence and system performance through the comparison with the other methods and schemes for the delay-sensitive applications and delay-tolerant applications.

• A grid-aware multi-agent reinforcement learning was developed for EV charging. • Public charging data worked with domestic demand to build a real environment. • A dual-head actor optimised the acceptance and cross-site redirection of EV charging. • A centralised attention-based critic and delayed replay buffer ensured stable learning. • The RL policy reduced peak load variability by 40 % and threshold violations by 37 %. As electric vehicles (EV) become more widespread, managing charging demand is critical for grid stability and efficient resource allocation. However, prior work often optimises single sites or treats demand as exogenous, without explicitly modelling grid capacity constraints or the aggregate effects of redirection, and it rarely offers an integrated capacity-aware view that links domestic and public charging under system-level coordination. This study, therefore, develops a multi-agent reinforcement learning framework with cross-site redirection to optimise EV charging operations. The research area focuses on the Perth and Kinross region of Scotland in the UK, using real public charging records from the council database as well as domestic charging statistics from the UK Department for Transport. Charging sites are modelled as cooperative agents using the Multi-Agent Deep Deterministic Policy Gradient algorithm, trained in a spatiotemporal environment that integrates public charging data and simulated domestic demand. We contribute by unifying public and home charging demand in one learning environment, introducing a cross-site actor trained under centralised training and decentralised execution, and incorporating a delayed replay buffer with short-horizon forecasts so redirection aligns with future congestion. The learned policy reduces peak-hour load standard deviation by up to 40 % and lowers cumulative threshold violations by 37 % compared to the baseline. Distinct weekday and weekend strategies emerge, enabling adaptive coordination under varying demand patterns. The study provides interpretable control for EV networks, balancing peak demand and service quality across sites while addressing the system-level coordination.