Active RIS-Assisted Uplink NOMA with MADDPG for Remote State Estimation in Wireless Sensor Networks.

In the fifth generation (5G) wireless communication network, to better offer service for different vertical industry applications, network slicing is needed to meet the differential requirements. In this paper, we study the joint problem of radio resource allocation and power resource allocation in the radio access network (RAN) to improve the radio resource utilization with multi-user demands in smart grid scenario. Different with conventional single-agent algorithm, we propose a non-orthogonal multiple access (NOMA) based multi-agent deep deterministic policy gradient (MADDPG) algorithm to solve this dynamic optimization problem. The scheme determines the remote radio unit (RRU) and physical resource block (PRB) allocation for each user, and updates the neural network to dynamically adjust the resource allocation strategy to maximize the long-term rate of network slices. The reliability and latency requirements of network slicing are guaranteed as well. Simulation results show the convergence and efficiency of the proposed algorithm.

The Dual Active Bridge converter (DABC), known for its bidirectional power transfer capability and high efficiency, plays a crucial role in various applications, particularly in electric vehicles (EVs), where it facilitates energy storage, battery charging, and grid integration. The Dual Active Bridge Converter (DABC), when paired with a high-performance CLLC filter, is well-regarded for its ability to transfer power bidirectionally with high efficiency, making it valuable across a range of energy applications. While these features make the DABC highly efficient, they also complicate controller design due to nonlinear behavior, fast switching, and sensitivity to component variations. We have used a Fractional-order PID (FOPID) controller to benefit from the simple structure of classical PID controllers with lower complexity and improved flexibility because of additional filtering gains adopted in this method. However, for a FOPID controller to operate effectively under real-time conditions, its parameters must adapt continuously to changes in the system. To achieve this adaptability, a Multi-Agent Reinforcement Learning (MARL) approach is adopted, where each gain of the controller is tuned individually using the Deep Deterministic Policy Gradient (DDPG) algorithm. This structure enhances the controller’s ability to respond to external disturbances with greater robustness and adaptability. Meanwhile, finding the best initial gains in the RL structure can decrease the overall efficiency and tracking performance of the controller. To overcome this issue, Grey Wolf Optimization (GWO) algorithm is proposed to identify the most suitable initial gains for each agent, providing faster adaptation and consistent performance during the training process. The complete approach is tested using a Hardware-in-the-Loop (HIL) platform, where results confirm accurate voltage control and resilient dynamic behavior under practical conditions. In addition, the controller’s performance was validated under a battery management scenario where the DAB converter interacts with a nonlinear lithium-ion battery. The controller successfully regulated the State of Charge (SOC) through automated charging and discharging transitions, demonstrating its real-time adaptability for BMS-integrated EV systems. Consequently, the proposed MARL-FOPID controller reported better disturbance-rejection performance in different working cases compared to other conventional methods.

Clusters of unmanned aerial vehicles (UAVs) are often used to perform complex tasks. In such clusters, the reliability of the communication network connecting the UAVs is an essential factor in their collective efficiency. Due to the complex wireless environment, however, communication malfunctions within the cluster are likely during the flight of UAVs. In such cases, it is important to control the cluster and rebuild the connected network. The asymmetry of the cluster topology also increases the complexity of the control mechanisms. The traditional control methods based on cluster consistency often rely on the motion information of the neighboring UAVs. The motion information, however, may become unavailable because of the interrupted communications. UAV control algorithms based on deep reinforcement learning have achieved outstanding results in many fields. Here, we propose a cluster control method based on the Decomposed Multi-Agent Deep Deterministic Policy Gradient (DE-MADDPG) to rebuild a communication network for UAV clusters. The DE-MADDPG improves the framework of the traditional multi-agent deep deterministic policy gradient (MADDPG) algorithm by decomposing the reward function. We further introduce the reward reshaping function to facilitate the convergence of the algorithm in sparse reward environments. To address the instability of the state-space in the reinforcement learning framework, we also propose the notion of the virtual leader–follower model. Extensive simulations show that the success rate of the DE-MADDPG is higher than that of the MADDPG algorithm, confirming the effectiveness of the proposed method.

Introduction To address the urgent need for enhanced grid flexibility in high-penetration distributed photovoltaic (PV) systems, this paper proposes a novel two-stage framework integrating multi-objective cluster partitioning and multi-agent reinforcement learning (MARL) for cooperative peak shaving. Methods First, a comprehensive multi-objective optimization model defines PV cluster partitioning using three domain-specific metrics: available peak shaving margin (APSM), peak shaving synergy (PSS), and node correlation degree (NCD). This approach replaces conventional clustering algorithms, ensuring partitions balance electrical compactness, regulation capacity, and operational coordination. Subsequently, each partitioned cluster acts as an autonomous agent in a hierarchical MARL framework. Leveraging the multi-agent deep deterministic policy gradient (MADDPG) algorithm, agents collaboratively optimize active/reactive power allocation among clusters while coordinating unit-level control within clusters. Results Validated on the IEEE-33 node system, the strategy achieves a 29.39% reduction in peak-to-valley difference and 54.44% improvement in power balance deviation, significantly smoothing net load curves and enhancing voltage stability. Discussion This work demonstrates how MARL-driven coordination unlocks the inherent flexibility of distributed PV clusters, providing a scalable solution for grid-edge resource participation in system-level peak shaving.

Piezoelectric inkjet printing has important applications in high-precision manufacturing, but precise control of droplet quality faces challenges such as the nonlinear coupling between drive waveform parameters and droplet characteristics, as well as environmental disturbances. To address these issues, this paper proposes a closed-loop droplet control method based on a curriculum deep deterministic policy gradient (DDPG) algorithm, with the standard deviation of droplet volume and velocity as the main optimization objectives. To tackle the complex mapping relationships and boost training efficiency, we first built a high-fidelity simulation environment using a four-layer deep residual network trained on real-world industrial ink droplet data. Following that, we designed a dynamic weight-adaptive composite reward mechanism and integrated a curriculum learning strategy with the Deep Deterministic Policy Gradient (DDPG) algorithm. Relying on our self-developed droplet observation platform, we achieve closed-loop control of the standard deviation of droplet volume and velocity. Experimental results show that this method can precisely control the droplet volume within ±0.1 pl of the target value, while keeping the velocity standard deviation below 40 mm/s. The adjustment process is stable without severe oscillations, and the method demonstrates strong robustness against environmental disturbances such as ink supply pressure. This provides an efficient and practical solution for the automated control of droplet quality in piezoelectric inkjet printing.

On performance of multi-RIS assisted multi-user nonorthogonal multiple access system over Nakagami-[formula omitted] fading channels

Bio-inspired distributed load frequency control in Islanded Microgrids: A multi-agent deep reinforcement learning approach

Vehicular networks face significant challenges in resource allocation due to heterogeneous Quality-of-Service (QoS) requirements and dynamic channel conditions. This paper proposes an efficient resource allocation framework for heterogeneous vehicular networks to address diverse QoS requirements in dynamic environments. We introduce a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm enhanced by Graph Attention Networks (GATs), named MADDPG-GAT, to optimize sub-bands allocation and power control for Vehicle-to-Vehicle (V2V) links, while minimizing interference to Vehicle-to-Infrastructure (V2I) links. By constructing a hierarchical relationship graph to capture intra-group and inter-group interactions among vehicles, the proposed MADDPG-GAT leverages local and global embeddings for cooperative decision-making. The framework employs centralized offline training for efficient policy learning and decentralized online execution for real-time decision-making with low overhead, with normalized reward function to enhance convergence and training stability. Simulation results demonstrate that our approach outperforms baseline methods in terms of system performance and robustness.

The applications of the deep reinforcement learning method to achieve the arcs welding by multi-robot systems are presented, where the states and the actions of each robot are continuous and obstacles are considered in the welding environment. In order to adapt to the time-varying welding task and local information available to each robot in the welding environment, the so-called multi-agent deep deterministic policy gradient (MADDPG) algorithm is designed with a new set of rewards. Based on the idea of the distributed execution and centralized training, the proposed MADDPG algorithm is distributed. Simulation results demonstrate the effectiveness of the proposed method.

With the development of the IoT (Internet of Things), sensors networks can bring a large amount of valuable data. In addition to be utilized in the local IoT applications, the data can also be traded in the connected edge servers. As an efficient resource allocation mechanism, the double auction has been widely used in the stock and futures markets and can be also applied in the data resource allocation in sensor networks. Currently, there usually exist multiple edge servers running double auctions competing with each other to attract data users (buyers) and producers (sellers). Therefore, the double auction market run on each edge server needs efficient mechanism to improve the allocation efficiency. Specifically, the pricing strategy of the double auction plays an important role on affecting traders’ profit, and thus, will affect the traders’ market choices and bidding strategies, which in turn affect the competition result of double auction markets. In addition, the traders’ trading strategies will also affect the market’s pricing strategy. Therefore, we need to analyze the double auction markets’ pricing strategy and traders’ trading strategies. Specifically, we use a deep reinforcement learning algorithm combined with mean field theory to solve this problem with a huge state and action space. For trading strategies, we use the Independent Parametrized Deep Q‐Network (I‐PDQN) algorithm combined with mean field theory to compute the Nash equilibrium strategies. We then compare it with the fictitious play (FP) algorithm. The experimental results show that the computation speed of I‐PDQN algorithm is significantly faster than that of FP algorithm. For pricing strategies, the double auction markets will dynamically adjust the pricing strategy according to traders’ trading strategies. This is a sequential decision‐making process involving multiple agents. Therefore, we model it as a Markov game. We adopt Multiagent Deep Deterministic Policy Gradient (MADDPG) algorithm to analyze the Nash equilibrium pricing strategies. The experimental results show that the MADDPG algorithm solves the problem faster than the FP algorithm.

Misinformation in online social networks (OSNs) has been an ongoing problem, and it has been studied heavily over recent years. In this article, we use gamification to tackle misinformation propagation in OSNs. First, we construct a game based on the notion of cooperative games on graphs where the nodes of the social network are players. We use random regular networks and real networks in our simulations to show that the constructed game follows evolutionary dynamics and that the outcome of the game depends on the relation between the structural properties of the network and the benefit and cost variables defined in a cooperative game. Second, we create a game on the network level where the players control a set of nodes. We define agents whose goal is to maximize the total reward that we set up to be the number of nodes affected at the end of the game. We propose a deep reinforcement learning (RL) technique based on the multiagent deep deterministic policy gradient (MADDPG) algorithm. We test the proposed method along with well-known node selection algorithms and obtain promising results on different social networks.

In this paper, we apply deep reinforcement learning (DRL) to solve the flocking control problem of multi-robot systems in complex environments with dynamic obstacles. Starting from the traditional flocking model, we propose a DRL framework for implementing multi-robot flocking control, eliminating the tedious work of modeling and control designing. We adopt the multi-agent deep deterministic policy gradient (MADDPG) algorithm, which additionally uses the information of multiple robots in the learning process to better predict the actions that robots will take. To address the problems such as low learning efficiency and slow convergence speed of the MADDPG algorithm, this paper studies a prioritized experience replay (PER) mechanism and proposes the Prioritized Experience Replay-MADDPG (PER-MADDPG) algorithm. Based on the temporal difference (TD) error, a priority evaluation function is designed to determine which experiences are sampled preferentially from the replay buffer. In the end, the simulation results verify the effectiveness of the proposed algorithm. It has a faster convergence speed and enables the robot group to complete the flocking task in the environment with obstacles.

Buildings account for over 70% of the electricity use in the US. As cities grow, high peaks of electricity consumption are becoming more frequent, which leads to higher prices for electricity. Demand response is the coordination of electrical loads such that they react to price signals and coordinate with each other to shave the peaks of electricity consumption. We explore the use of multi-agent deep deterministic policy gradient (DDPG), an adaptive and model-free reinforcement learning control algorithm, for coordination of several buildings in a demand response scenario. We conduct our experiment in a simulated environment with 10 buildings.

Aerial base stations (AeBSs), as crucial components of air-ground integrated networks, are widely employed in cloud computing, disaster relief, and various applications. How to quickly and efficiently deploy multi-AeBSs for higher capacity gain has become a key research issue. In this paper, we address the 3D deployment optimization problem of multi-AeBSs with the objective of maximizing system capacity. To overcome communication overhead and privacy challenges in multi-agent deep reinforcement learning (MADRL), we propose a federated deep deterministic policy gradient (Fed-DDPG) algorithm for the multi-AeBS deployment decision. Specifically, a high-altitude platform (HAP)-assisted multi-AeBS deployment architecture is designed, in which low-altitude AeBS act as the local nodes to train its own deployment decision model, while the HAP acts as the global node to aggregate the weights of local models. In this architecture, AeBSs do not exchange raw data, addressing data privacy concerns and reducing communication overhead. Simulation results show that the proposed algorithm outperforms fully distributed MADRL algorithms and closely approximates the performance of multi-agent deep deterministic policy gradient (MADDPG), which requires global information during training, but with less training time.

In this article, we explore how to optimize task allocation for robot swarms in dynamic environments, emphasizing the necessity of formulating robust, flexible, and scalable strategies for robot cooperation. We introduce a novel framework using a decentralized partially observable Markov decision process (Dec-POMDP), specifically designed for distributed robot swarm networks. At the core of our methodology is the local information aggregation multiagent deep deterministic policy gradient (LIA-MADDPG) algorithm, which merges centralized training with distributed execution. During the centralized training phase, a local information aggregation (LIA) module is meticulously designed to gather critical data from neighboring robots, enhancing decision-making efficiency. In the distributed execution phase, a strategy improvement method is proposed to dynamically adjust task allocation based on changing and partially observable environmental conditions. Our empirical evaluations show that the LIA module can be seamlessly integrated into various centralized training and decentralized execution (CTDE)-based multiagent reinforcement learning (MARL) methods, significantly enhancing their performance. Additionally, by comparing LIA-MADDPG with six conventional reinforcement learning algorithms and a heuristic algorithm, we demonstrate its superior scalability, rapid adaptation to environmental changes, and ability to maintain both stability and convergence speed. These results underscore LIA-MADDPG's outstanding performance and its potential to significantly improve dynamic task allocation in robot swarms through enhanced local collaboration and adaptive strategy execution.