Intelligent Game-Theoretic Deep Reinforcement Learning Caching for Named Data Networking: A Unified Optimization Framework

With the increasing real-time demand in the internet era, especially dynamic requests for last-mile delivery, route planning is becoming more computationally expensive than ever before. Many supply chains (SCs) choose the joint distribution of multiple depots to reduce transportation costs and delivery times. However, providing real-time and high-quality solutions for such complex routing problems remains challenging. Current solution methods like mathematical programming and heuristics suffer from scalability issues and long computation times. In contrast, artificial intelligence, especially Deep Reinforcement Learning (DRL), provides a general-purpose framework for sequential decision-making that has produced good results for many challenging real-life problems. However, applying DRL to route multiple vehicles is nontrivial, as the joint distribution requires an effective method that facilitates collaboration and communication among all agents while they carry out the delivery mission. In this research, a collaborative Multi-Agent Deep Reinforcement Learning (MADRL) approach is proposed for routing multiple vehicles in the SC. The proposed MADRL model leverages the power of two frameworks, deep learning and reinforcement learning, to generate routing policies for all agents in real time. Experimental results show the ability of the proposed learning model to obtain fast and quality solutions for complex delivery problems. Furthermore, the generalization ability of MADRL is also validated by testing the well-trained model on different scale problems.

With the improvement in the intelligence level of UAVs and the development of cluster control technology, the intelligent decision-making method of UAV cluster confrontation will become the key technology of UAV combat in the future. The UAV cluster confrontation learning environment is of complex characteristics of high dimension, non-linearity, incomplete information, continuous action space, and so on. Recently, artificial intelligence technology represented by deep learning and reinforcement learning has made a great breakthrough. Deep reinforcement learning has shown a great ability to solve intelligent decision-making problems in complex environments. In this paper, enlightened by the multi-agent centralized training distributed execution framework and the idea of maximum policy entropy, we propose a deep reinforcement learning method of the multi-agent soft actor-critic (MASAC) with incomplete information. A game model for UAV cluster confrontation based on multi-agent deep reinforcement learning is established, and a continuous space multiple UAV combat environment is constructed. Simulation experiments are performed on the asymmetric confrontation of UAV clusters with red and blue teams. The experimental results show that MASAC outperforms the existing popular multi-agent deep reinforcement learning methods, making the game players converge to a higher return equilibrium point of the game. Moreover, the convergence of MASAC is investigated and analyzed extensively, and the results show that MASAC is of good convergence and stability, ensuring the practicability of MASAC in intelligent decision-making regarding UAV cluster confrontation.

Deep reinforcement learning (DRL) has superior autonomous decision-making capabilities, combining deep learning and reinforcement learning (RL). Unlike DRL employs deep neural networks (DNNs), broad RL (BRL) adopts the broad learning system (BLS) that is established with flat networks to generate the strategy. This article proposes the multiagent adaptive broad-DRL (ABDRL) approach for traffic light control (TLC), which combines the broad network with the deep network structure. Specifically, the structure of ABDRL first expands in the form of flatted broad networks. Then, the feature representation module that contains DNNs is employed to extract the critical traffic information. In addition, experiences sampled randomly by the experience replay mechanism cannot reflect the current training status of the agent effectively. In order to alleviate the impacts caused by random sampling, the forgetful experience mechanism (FEM) is incorporated into ABDRL. The FEM enables the agent to discriminate the importance of experiences stored in the experience reply buffer to improve robustness and adaptability. We validate the effectiveness of ABDRL in TLC, and the results illustrate the optimality and robustness of ABDRL over the state-of-the-art multiagent DRL (MADRL) algorithms.

Attempts to address the production scheduling problem thus far rely on simplifying assumptions, such as static environment and inflexible size of the problem, which compromises the schedule performance in practice due to many unpredictable disruptions to the system. Thus, the study of scheduling in the presence of real-time events, termed dynamic scheduling, continues to attract attention given the agility, flexibility, and timeliness modern production systems must deliver. Additionally, the changing nature of the manufacturing system also raises new challenges to existing scheduling strategies. At the front-end, the development of advanced data creation and exchange frameworks such as the Internet of things and cyber-physical system and their applications to the industrial environment have created an abundance of industrial data, while at the backend, edge and cloud computing technologies greatly enhance the capacity to process that data. Industrial data must be mined and analyzed so that the investment in infrastructure is not wasted, and the production system managed more effectively and in real-time. Many data-driven technologies have been adopted in scheduling research, a promising candidate among them being reinforcement learning (RL) which is able to build a direct mapping from observation of environment to actions that improve its performance. In this thesis, a deep multi-agent reinforcement learning (deep MARL) architecture is proposed to solve the dynamic scheduling problem (DSP). The deep reinforcement learning (DRL) algorithm is used to train the decentralized scheduling agents, to capture the relationship between information on the factory floor and scheduling objectives, with the aim of making real-time decisions for a manufacturing system with frequent unexpected events. Two major aspects of deep MARL application to DSP are addressed in this work, namely the conversion from traditional static scheduling problem (SSP) to dynamic scheduling in a practical context, and the adaptation of existing deep MARL algorithms to solve the scheduling problem in such an environment. Some impractical constraints of traditional studies are removed to create a research context that is closer to actual practice, result in a scheduling problem of variable size and scope. Specialized state and action representations that can handle the ever-changing specification of problem are developed; the criteria of feature selection in dynamic environment are also discussed. Recent progressions in DRL and MARL research are integrated into the proposed approach after selection and adaptation. In addition, various improvements to common deep MARL architecture are proposed, including the lightweight multilayer perceptron (MLP) encoder that is efficient in handling unstructured industrial data, a training scheme under the multi-agent architecture to improve the stability of training and overall performance, and knowledge-based reward-shaping techniques to decompose the joint reward signal into individual utilities to speed up the learning and encourage cooperative behavior between agents. Simulation studies are then conducted for the ablation study and validation. In the first stage, the performance of the proposed approach, either as individual components or as an integrated model, are tested in iterative simulation runs within which a unique instance of production is created. Meanwhile, a set of DRL-based approaches from recent publications are run in parallel. Results suggest that the contribution of each improvement is significant; the integrated architecture also delivers stronger performance than peer DRL-based approaches. For the validation, a set of priority rules that have strong performance in specified context and are widely applied in actual production scheduling are used as the benchmark. Proposed approach also provides performance gain compared to the strongest rule, with a minor increase in computation cost and negligible latency in decision-making.

A multi-agent deep reinforcement learning based energy management for behind-the-meter resources

The spectrum sharing design (such as the underlay sidelink configuration in cellular networks) for vehicle-to-vehicle (V2V) and vehicle-to-network (V2N) links can improve the efficiency of spectrum utilization for vehicular networks. However, differentiated service requirements for V2V and V2N links and the greater complexity of interference management make it challenging for conventional centralized learning-based methods in the face of real-world deployment. This motivates the exploration of multi-agent deep reinforcement learning (DRL) solutions, while existing research has predominantly focused on cooperative setting that causes large communication overheads for real-world deployment. This paper proposed a decentralized resource allocation scheme for joint spectrum sub-band and V2V transmission power selection that utilizes federated learning (FL) to formulate a mixed setup to avoid frequent communication and enable easy model sharing, termed FL-MX for the design of FL and mixed DRL agent cooperation level. Different levels of cooperation among agents are also explored by implementing different state-of-the-art (SOTA) DRL-based resource allocation algorithms on the developed standard-compliant simulation platform. Through extensive simulations, it is observed that the proposed FL-MX manages to converge effectively and can match the best SOTA performance, gaining a 22.55% and 9.5% improvement in the V2V payload delivery ratio and V2N throughput compared to the random baseline. Such results make it promising for future research to investigate multi-agent DRL designs at a mixed agent cooperation level for high-performing network decision solutions with less frequent communication demands.

Mobile edge computing (MEC) has emerged as a key enabler for low-latency and computation-intensive applications. However, conventional terrestrial MEC networks face challenges in coverage, scalability, and dynamic environments. To address these limitations, this paper explores unmanned aerial vehicle (UAV)-assisted MEC, leveraging the mobility and flexibility of UAVs to enhance edge computing performance. However, challenges such as dynamic mobile user association, multi-UAV collaborative trajectory design, and limited communication and computation resources remain significant hurdles. We then formulate a joint optimization problem for user association, trajectory design, and resource allocation for the multi-UAV collaborative MEC network, aiming to minimize system cost considering computation task demands and UAV resource constraints. The problem, modeled as a Mixed Integer NonLinear Program (MINLP), is reformulated as a decentralized partially observable Markov decision process (Dec-POMDP) to capture complex system dynamics. Afterwards, we propose a multi-agent generative-enhanced deep reinforcement learning method that integrates generative diffusion model (GDM) with multi-agent deep reinforcement learning (MADRL) to effectively handle high-dimensional action spaces and enable collaborative multi-UAV decision-making. Extensive simulations using a real-world mobility dataset demonstrate that the proposed method consistently outperforms three baselines in terms of time-average system cost, task completion delay, task satisfactory ratio, and energy consumption ratio when varying the number of UAVs.

Different types of construction equipment engage in complex cooperative interactions on construction sites, significantly influencing project progress. This study addresses the problem of cooperative excavation involving multiple heterogeneous machines—such as excavators and bulldozers—within a multi-agent Deep Reinforcement Learning (DRL) framework. A parameter-sharing Proximal Policy Optimization (PPO) architecture is introduced in a grid-world simulation environment, where all agents share a unified policy network and learn coordinated strategies through end-to-end training. To promote effective collaboration and address exploration challenges, a dense reward structure is designed, incorporating incentives for task completion, spoil delivery, excavation chain formation, and depth progression. Experimental results demonstrate that the proposed method achieves full cooperation within 2 minutes in small environments. As the scale increases, runtime grows approximately linearly and completion drops to 0.85.

Deploying low-altitude unmanned aerial vehicles (UAVs) as aerial base stations (ABSs) is a promising approach to enhance cellular network coverage in dynamic and infrastructure-constrained environments. However, real-world UAV deployment involves complex challenges, including managing partially observable and mobile users, maintaining multi-hop backhaul connectivity to a fixed ground base station, mitigating co-channel interference via minimum UAV separation, and optimizing coverage efficiency, fairness, and energy consumption. This paper proposes a multi-agent deep reinforcement learning (MARL) framework that formulates the dynamic UAV deployment task as a centralized multi-input multi-output policy. Our method employs sequential action masking to guarantee multi-hop connectivity and enforce minimum distance constraints during UAV placement. The policy network integrates graph neural networks, residual convolutional layers, and feature-wise linear modulation to effectively capture spatial relations, inter-UAV interactions, and global and local network states. We validate our approach through extensive simulations across various user density scenarios, demonstrating significant improvements in coverage efficiency, fairness of coverage distribution, and flight energy reduction compared to strong baseline algorithms. Our proposed framework provides a comprehensive solution that simultaneously addresses dynamic user mobility, multi-hop connectivity, interference mitigation, and energy efficiency in UAV-assisted cellular networks.

The rise of the artificial intelligence (AI) brings golden opportunity to accelerate the development of the intelligent transportation system (ITS). The platoon control of connected autonomous vehicle (CAV) as the key technology exhibits superior for improving traffic system. However, there still exist some challenges in multi-objective platoon control and multi-agent interaction. Therefore, this paper proposed a connected autonomous vehicle latoon control approach with multi-agent deep reinforcement learning (MADRL). Finally, the results in stochastic mixed traffic flow based on SUMO (simulation of urban mobility) platform demonstrate that the proposed method is feasible, effective and advanced.KeywordsIntelligent transportation systemConnected autonomous vehicleMulti-objective platoon controlMulti-agent deep reinforcement learning

Due to the complexity of three-dimensional (3D) human pose, it is difficult for ordinary sensors to capture subtle changes in pose, resulting in a decrease in the accuracy of 3D human pose detection. A novel 3D human motion pose detection method is designed by combining Nano sensors and multi-agent deep reinforcement learning technology. First, Nano sensors are placed in key parts of the human to collect human electromyogram (EMG) signals. Second, after de-noising the EMG signal by blind source separation technology, the time-domain and frequency-domain features of the surface EMG signal are extracted. Finally, in the multi-agent environment, the deep reinforcement learning network is introduced to build the multi-agent deep reinforcement learning pose detection model, and the 3D local pose of the human is output according to the features of the EMG signal. The fusion and pose calculation of the multi-sensor pose detection results are performed to obtain the 3D human pose detection results. The results show that the proposed method has high accuracy for detecting various human poses, and the accuracy, precision, recall and specificity of 3D human pose detection results are 0.97, 0.98, 0.95 and 0.98, respectively. Compared with other methods, the detection results in this paper are more accurate, and can be widely used in medicine, film, sports and other fields.

Dear Editor, This letter presents a new transfer learning framework for the deep multi-agent reinforcement learning (DMARL) to reduce the convergence difficulty and training time when applying DMARL to a new scenario [1], [2]. The proposed transfer learning framework includes the design of neural network architecture, curriculum transfer learning (CTL) and strategy distillation. Experimental results demonstrate that our framework enables DMARL models to converge faster while improving the final performance.

Maintaining high-performance operation under dynamic and nonuniform network traffic has been a technical challenge in current data center networks (DCNs). With the aim to provide better quality of service (QoS) for diverse applications, this work presents a dynamic and adaptive DCN reconfiguration framework based on deep reinforcement learning (DCR2L). The proposed framework is integrated into the SDN control plane of the DCN, implementing real-time and automatic DCN reconfiguration. Performance of the DCR2L framework is experimentally demonstrated in our DCN lab, including 4 racks and 16 servers. Experimental results show a network latency improvement of 6.9% based on the DCR2L at an average network bandwidth of 2.3 Gb/s. Based on measured traffic and physical parameters in the experiment, performance of the DCR2L framework is numerically assessed with the realistic traffic of diverse QoS requirements for both electrical and optical DCNs and for different data center scales. Leaf–spine electrical and OPSquare optical networks are set up in an OMNeT++ simulator. For a data plane network consisting of 16 racks and 320 servers, results indicate that the DCR2L framework improves the network latency of up to 16.4% under leaf–spine and up to 24.6% under OPSquare for the overall traffic with respect to the classical heuristic method. For a DCN scale of 10,240 servers, the DCR2L framework provides up to 12.5% lower latency for the leaf–spine electrical network and up to 17.8% latency improvement for the OPSquare optical network.

This work proposes a new model of reconfigurable intelligent surface (RIS) called cognizable RIS (CRIS) that is specifically designed to meet the unique demands of users who require extreme-ultra-reliable and low-latency Communication (xURLLC) in the sixth generation (6G) wireless networks. The programmable elements in the proposed CRIS unit can adapt to different modes of operation to provide significant performance gain. To improve reliability at the receiver, we integrate unmanned aerial vehicles with the CRIS module, which enhances network performance through beamforming and mobility. Our study focuses on maximizing the sum throughput in a multiple-input multiple-output scenario using the rate-splitting multiple access communication system. To achieve this, we introduce a novel hybridized multi-agent-based deep reinforcement learning (DRL) algorithm for optimal resource allocation that maximizes the sum throughput. We incorporate long-short-term memory (LSTM) networks into our proposed DRL to address the temporal dependencies due to stochastic channel conditions. By utilizing the proposed LSTM-based multi-agent DRL (MA-DRL) algorithm, we achieve notable gains of 11.7% and 26.9% in sum throughput over widely recognized DRL benchmark algorithms, all while adhering to xURLLC’s stringent maximum packet error probability constraint of 10−9.

The rapid growth of connected medical devices generates massive volumes of heterogeneous health data that must be processed and transmitted in real time. In such environments, minimizing latency and energy consumption remains a critical challenge for next-generation health monitoring systems. Existing reinforcement learning and optimization methods for intelligent communication networks face several challenges, including slow convergence, high computational overhead, and inefficiency in handling task prioritization. To resolve these issues, this work develops a chaotic dung beetle optimization-boosted multi-agent deep reinforcement learning that jointly optimizes communication reliability, computational efficiency, and task prioritization. A reward function is designed to jointly minimize delay, energy usage, and system cost while preserving information freshness. Specifically, the dung beetle optimization process is combined with a piecewise linear chaotic map to enhance population diversity, which significantly improves search space exploration and leads to faster convergence and higher solution quality. The proposed algorithm enhances the exploration capability of multi-agent deep reinforcement learning through the integration of chaotic dung beetle optimization, enabling more accurate and reliable decision-making in real-world applications. Extensive experiments demonstrate that the proposed chaotic dung beetle optimization-boosted multi-agent deep reinforcement learning model achieves superior performance compared to baseline algorithms. Specifically, it reaches an accuracy of over 97.00% with rapid convergence, reduces system cost under varying health data sizes and Medical Internet of Things devices, and maintains robust scalability across diverse workloads. Moreover, the model achieves significant reductions in communication latency and energy consumption as central processing unit cycles and bandwidth increase, while effectively prioritizing high-criticality tasks.