A Deep Reinforcement Learning Approach With Attention Mechanism for DAG Task Scheduling in Data Centers

Energy efficient task scheduling based on deep reinforcement learning in cloud environment: A specialized review

Optimal scheduling of island integrated energy systems considering multi-uncertainties and hydrothermal simultaneous transmission: A deep reinforcement learning approach

Millimeter-wave (mmWave) communication is a key technology of fifth generation wireless systems to achieve the expected 1000x data rate. With large bandwidth at the mmWave band, the link capacity between users and base stations (BSs) can be much higher compared to sub-6 GHz wireless systems. Meanwhile, due to the high cost of infrastructure upgrade, it would be difficult for operators to drastically enhance the capacity of backhaul links between mmWave BSs and the core network. As a result, the data rate provided by backhaul may not be sufficient to support all mmWave links; hence, the backhaul connection becomes the new bottleneck. On the other hand, as mmWave channels are subject to random blockage, the data rates of mmWave users significantly vary over time. With limited backhaul capacity and highly dynamic data rates of users, how to allocate backhaul resource to each user remains a challenge for mmWave systems. In this article, we present a deep reinforcement learning (DRL) approach to address this challenge. By learning the blockage pattern, the system dynamics can be captured and predicted, resulting in efficient utilization of backhaul resource. We begin with a discussion on DRL and its application in wireless systems. We then investigate the problem of backhaul resource allocation and present the DRL-based solution. Finally, we discuss open problems for future research and conclude this article.

This paper investigates the covert communications via spectrum allocations in a cellular-enabled unmanned aerial vehicle (UAV) network consisting of a base station (BS), UAVs, ground users (GUs), and a warden, where warden attempts to detect the transmission from a target GU to a UAV receiver. We formulate the spectrum allocation as an optimization problem with the constraints of covertness performance requirement and the qualities of service (QoS) of cellular communications. This is a nonlinear and nonconvex problem, which is generally challenging to be solved. Thus, we propose a deep reinforcement learning (DRL) approach to solve it. Under such an approach, we first model the multi-agent DRL environment in such networks. Then we define the state, action, reward and interaction mechanism of the DRL environment. Finally, a DRL algorithm is presented for learning the optimal policy of spectrum allocation.

The modern network is equipped with many service functions to acquire high-quality service. The emergence of network function virtualization (NFV) provides a convenient way to accomplish the network services in the form of virtual network function (VNF) and also makes the scheduling solution of VNFs flexible. The VNFs can be deployed on commodity servers as software processes. Besides, multiple VNFs are chained in a specified order as a service function chain (SFC) to serve a given flow, increasing the scheduling difficulty to minimize the average flow completion time. In this paper, we study the problem of scheduling multiple SFCs with the constraint of different resource limitations in various commodity servers. This problem is typically formulated as an Integer Linear Programming (ILP) problem, which is NP-hard. To well tackle this problem, we propose a deep reinforcement learning (DRL) approach. It involves multi-step decision making, which can be naturally transformed into a DRL problem. We design specific reward and state representations for such a multi-resource scheduling problem. We also consider how to use DRL to handle online requests of SFCs. The experiment results demonstrate that the DRL approach can significantly reduce the average completion time of a set of SFC and achieves a cost saving of 39.94% against the benchmark method.

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

Dynamic pricing, which aims to dynamically adjust the charging price in a timely fashion to unlock the flexibility of electric vehicle (EV) customers, has been extensively studied with the rapid development of charging technologies. Many existing works on dynamic pricing focus on maximizing the social welfare of charging service providers and EV customers. Cases of high-dimensional charging environments, which are often encountered with the rapid growth of EV market penetration, have been rarely considered to date. This article proposes a new dynamic pricing framework for EV charging stations that can offer multiple charging options to customers over a finite-time horizon. The charging price can be dynamically adjusted to maximize the quality of service (QoS) with a differentiated service requirement level (SRL) whenever the arrival rates and queuing system capacities of the charging systems are given at the end of a time period. The dynamic pricing problem is formulated as a finite-discrete horizon Markov decision process (MDP) with a mixed state space. A customized deep reinforcement learning (DRL) approach is employed to solve the examined EV dynamic pricing problem. The simulation results demonstrate the effectiveness of the proposed method.

The scheduling of parallel real-time tasks on multiprocessor systems is more complicated than the one of independent sequential tasks, specially for the Directed Acyclic Graph (DAG) model. The complexity is due to the structure of DAG tasks and the precedence constraints between their subtasks. The trivial DAG scheduling approach is to directly apply common real-time scheduling algorithms on DAGs despite their lack of compatibility with the parallel model. Another scheduling approach, which is called the stretching method, aims at transforming each parallel DAG task in the set into a collection of independent sequential threads that are easier to be scheduled. In this paper, we are interested in analyzing global preemptive scheduling of DAGs using both approaches by showing that they are not comparable when associated with Deadline Monotonic (DM) and Earliest Deadline First (EDF) scheduling algorithms. Then we use extensive simulations to evaluate their schedulability performance. To this end, we use our simulation tool YARTISS to generate random DAG tasks with many parameter variations so as to guarantee reliable experimental results.

Multi-agent quantum-inspired deep reinforcement learning for real-time distributed generation control of 100% renewable energy systems

Task assignment is a challenging problem in multiple unmanned aerial vehicle (UAV) missions. In this paper, we focus on the task assignment problem for a UAV swarm saturation attack, in which a deep reinforcement learning (DRL) framework is developed. Specifically, we first construct a mathematical model to formulate the task assignment problem for a UAV swarm saturation attack and consider it as a Markov Decision Process (MDP). We then design a policy neural network using the attention mechanism. We also propose a training algorithm based on the policy gradient method so that our agent can learn an effective task assignment policy. The experimental results have shown that our DRL method can generate high-quality solutions for different problem scales, which meets the requirements of real-time and flexibility in the actual situation.

The rapid expansion of data centers (DCs) has led to substantial increases in global energy consumption and carbon emissions. Moreover, the strong coupling among workload scheduling, IT, cooling, and energy subsystems makes sustainable DC microgrids highly complex, especially given the difficulty of accurate modeling and the presence of significant uncertainties and rapid dynamics. To address these challenges, this paper proposes a multi-critic deep reinforcement learning (DRL) approach based on a mixture-of-experts (MoE) with multi-head architecture. The problem is formulated as a multi-reward Markov Decision Process (MDP) with independent reward signals for workload efficiency, energy consumption, and carbon footprint, providing richer feedback compared to traditional single-reward formulations. In the proposed approach, each reward is assigned a dedicated critic, thereby avoiding the conflicts and interference that arise when a single critic attempts to learn multiple competing rewards simultaneously. The shared MoE foundation with the multi-head architecture enables each critic head to focus on learning the value function for its specific optimization target, while the integrated adaptive gating mechanisms facilitate dynamic leveraging of both common and task-specific knowledge. This architecture improves learning stability, accelerates convergence, and enhances adaptability to complex, multi-objective learning tasks. Extensive experiments on a DC microgrid show that the proposed method outperforms state-of-the-art DRL and rule-based baselines in reducing the overall energy consumption and carbon emissions. All codes can be found at https://github.com/ikelq/Mixture-of-experts-based-Multi-Critic-DRL-for-Sustainable-Management-of-Data-Center-Microgrids . • Formulate the sustainable management of a DC microgrid as a multi-reward MDP. • Propose a multi-critic DRL framework with each reward assigned to a dedicated critic. • Design a multi-head architecture with a shared MoE structure within the multi-critic DRL framework.

Given a project, the team formation problem (TFP) aims to find a team of experts with all the required skills and minimize the team’s communication cost. The TFP has emerged as a prominent research topic due to its substantial influence on team performance and the subsequent impact on organizational success. Previous research mainly concentrates on proposing heuristic or metaheuristic algorithms, yet fails to consider the historical experience of solving previous TFP. In addition, most existing learning-based approaches rely on complex multistage workflows, rather than learning the entire process end-to-end. This article proposes an end-to-end deep reinforcement learning approach that focuses on effectively reusing previous experience to tackle the TFP. First, we apply the Markov decision process to model the team formation process, in which the state is primarily characterized by the currently formed team and the skill requirements of the project. Second, we propose a self-updating deep reinforcement learning (SURL) model, in which the social network information serves as the input for the static encoding module, while the skills’ experts and the skill requirements of the project act as the inputs for the dynamic encoding module. Next, a novel deep reinforcement learning algorithm is designed to train the model by fully leveraging the historical experience of team formation. Finally, comprehensive experiments with real-world datasets demonstrate that our proposed model outperforms several commonly used optimization algorithms in terms of the average communication cost.

Failure-resilient DAG task scheduling in edge computing

Deep Reinforcement Learning for energy-aware task offloading in join SDN-Blockchain 5G massive IoT edge network

With the spread of online platforms, problems such as manned/unmanned food delivery, cargo delivery, raw material delivery, are increasing the importance of logistics day by day. Vehicle routing problem, which is one of the most important problems in the field of logistics, is a combinatorial problem and as the problem space grows, it takes a long time to find a solution with human effort and in most cases it is not even possible. Thus, it becomes essential for the solution of this problem to be autonomous. Although it is possible to solve the problem with classical heuristic optimization methods, it takes a long time and sometimes does not give a good enough solution. Deep reinforcement learning models with attention mechanisms have great potential in this regard. However,in case of insufficient training in large problem space, it is possible to get away from the optimal solution. In this study, better results are taken in an acceptable time by using the deep reinforcement learning models with attention-model and heuristic methods in a hybrid way.