Deep Reinforcement Learning based UAV-Assisted Maritime Network Computation Offloading Strategy

Collaborative computation offloading and wireless charging scheduling in multi-UAV-assisted MEC networks: A TD3-based approach

This paper considers the computation offloading and resource allocation for system sum cost minimization in a mobile edge computing (MEC)network. The computation offloading decision of the user equipments (UEs)can be made through wireless link to access the MEC server. Our intent is to minimize the weighted sum cost of the system, which takes into account the energy consumption and time delay, for both task processing and data transmissions, subject to the force delay requirement of the devices. We first formulate the problem as a mixed-integer and non-convex optimization scheme. Enlightened by the superior performance of reinforcement learning (RL)on solving resource control problems, we put forward a novel RL-based scheme for computation offloading and resource allocation in MEC network. Specifically, the MEC node is considered as an “agent”, which first makes decision on whether to offloading the computing task to the MEC server, and then utilizes convex optimization algorithms to settle the radio spectrum and computation resource scheduling problem in each decision phase. The simulation results present that the RL-based method can achieve better computation offloading and resource allocation performance compared to those of local computing and remote computing modes. It also achieves very similar or slightly better performance compared to the genetic algorithm based method.

This chapter studies mobile edge computing (MEC) networks assisted by unmanned aerial vehicles (UAVs). According to the application scenarios, we consider three roles for UAVs in MEC networks: exploiting MEC computing capabilities, serving as a computing server, and serving as a relay for computation offloading. Furthermore, the details for resource allocation and optimization are presented in the three scenarios of UAV-assisted MEC networks. In addition, we focus on the situation in which a UAV not only functions as an MEC server to inspect turbines on a wind farm, but also performs task computation. To facilitate wide applications of UAV-assisted MEC in practice, this chapter highlights the main implementation issues of UAV-assisted MEC, including optimal UAV deployment, wind models, and joint trajectory–computation performance optimization.

Resource allocation in RISs-assisted UAV-enabled MEC network with computation capacity improvement

Using Unmanned aerial vehicles (UAVs) to assist mobile edge computing (MEC) network is a promising solution to provide flexible and low-latency computing service for resource limited user equipments (UEs). However, the computing resources that one UAV can provide are often limited and cannot provide computing services for massive mobile users, thus limiting the application of this technology. So, a novel multi-UAV-assisted MEC network is proposed in this paper. The optimization problem is formulated with the aim to make full use of network's computing resources to reduce the number of failed tasks and energy consumption. To solve this challenging problem, a novel solution is proposed. First, we use a swarm intelligence algorithm to investigate the deployment of UAVs. Second, we offer an efficient matching algorithm to explore the tasks assignments under the given deployment of UAVs. The simulation results show that the proposed solution can effectively reduce the number of failed tasks and energy consumption.

This article presents a cooperative multiagent deep reinforcement learning (MADRL) approach for unmanned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks. An UAV with computing capability can provide task offlaoding services to ground Internet of Things devices (IDs). With partial observation of the entire network state, the UAV and the IDs individually determine their MEC strategies, i.e., UAV trajectory, resource allocation, and task offloading policy. This requires joint optimization of decision-making process and coordination strategies among the UAV and the IDs. To address this difficulty, the proposed cooperative MADRL approach computes two types of action variables, namely, message action and solution action, each of which is generated by dedicated actor neural networks (NNs). As a result, each agent can automatically encapsulate its coordination messages to enhance the MEC performance in the decentralized manner. The proposed actor structure is designed based on graph attention networks such that operations are possible regardless of the number of IDs. A scalable training algorithm is also proposed to train a group of NNs for arbitrary network configurations. Numerical results demonstrate the superiority of the proposed cooperative MADRL approach over conventional methods.

Mobile Edge Computing (MEC) migrates the computing center to the network edge to provide computing services for Mobile Users (MUs). However, due to the limited capacity of MUs and insufficient types of services stored by edge servers, it brings great challenges to computation offloading and service caching in Internet of Things (IoT) network. In this paper, we investigate the joint optimization problem of computation offloading, resource allocation and service caching replacement in a collaborative MEC system to minimize the total cost of all MUs under the latency and resource constraints. Accordingly, we formulate the problem as a Mixed Integer Non-linear Programming (MINLP) problem. In order to solve this problem, the Deep Deterministic Policy Gradient (DDPG)-based algorithm is proposed. The simulation results show that the proposed algorithm is better than other benchmark schemes in different scenarios.

Mobile edge computing (MEC) has emerged as a key enabler for low-latency and computation-intensive applications. However, conventional terrestrial MEC networks face challenges in coverage, scalability, and dynamic environments. To address these limitations, this paper explores unmanned aerial vehicle (UAV)-assisted MEC, leveraging the mobility and flexibility of UAVs to enhance edge computing performance. However, challenges such as dynamic mobile user association, multi-UAV collaborative trajectory design, and limited communication and computation resources remain significant hurdles. We then formulate a joint optimization problem for user association, trajectory design, and resource allocation for the multi-UAV collaborative MEC network, aiming to minimize system cost considering computation task demands and UAV resource constraints. The problem, modeled as a Mixed Integer NonLinear Program (MINLP), is reformulated as a decentralized partially observable Markov decision process (Dec-POMDP) to capture complex system dynamics. Afterwards, we propose a multi-agent generative-enhanced deep reinforcement learning method that integrates generative diffusion model (GDM) with multi-agent deep reinforcement learning (MADRL) to effectively handle high-dimensional action spaces and enable collaborative multi-UAV decision-making. Extensive simulations using a real-world mobility dataset demonstrate that the proposed method consistently outperforms three baselines in terms of time-average system cost, task completion delay, task satisfactory ratio, and energy consumption ratio when varying the number of UAVs.

With the ongoing advancement of electric power Internet of Things (IoT), traditional power inspection methods face challenges such as low efficiency and high risk. Unmanned aerial vehicles (UAVs) have emerged as a more efficient solution for inspecting power facilities due to their high maneuverability, excellent line-of-sight communication capabilities, and strong adaptability. However, UAVs typically grapple with limited computational power and energy resources, which constrain their effectiveness in handling computationally intensive and latency-sensitive inspection tasks. In response to this issue, we propose a UAV task offloading strategy based on deep reinforcement learning (DRL), which is designed for power inspection scenarios consisting of mobile edge computing (MEC) servers and multiple UAVs. Firstly, we propose an innovative UAV-Edge server collaborative computing architecture to fully exploit the mobility of UAVs and the high-performance computing capabilities of MEC servers. Secondly, we established a computational model concerning energy consumption and task processing latency in the UAV power inspection system, enhancing our understanding of the trade-offs involved in UAV offloading strategies. Finally, we formalize the task offloading problem as a multi-objective optimization issue and simultaneously model it as a Markov Decision Process (MDP). Subsequently, we proposed a task offloading algorithm based on a Deep Deterministic Policy Gradient (OTDDPG) to obtain the optimal task offloading strategy for UAVs. The simulation results demonstrated that this approach outperforms baseline methods with significant improvements in task processing latency and energy consumption.

Unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) system is a prominent strategy where a UAV equipped with an MEC server is deployed to serve terminal devices. This paper considers a multi-UAV assisted network in which multiple UAVs and a terrestrial base station (BS) are deployed to provide MEC services to mobile users. The objective is to minimize an energy and latency-based cost function by jointly optimizing task offloading and MEC server selection decision, transmission power, UAV trajectory, and CPU frequency allocation. An alternating iterative approach based on the block descent method is proposed to solve this problem. In the first layer, task offloading and server selection decision subproblem is solved using a game theoretic approach. The second layer handles offloading and downloading transmission power allocations by utilizing a simplistic geometric waterfilling (GWF) technique, and the UAV trajectory by successive convex approximation (SCA). Whereas, the third layer solves the computation resource subproblem by performing CPU frequency allocation using a gradient descent method. The proposed method uses a segment-by-segment approach, which divides the entire UAV flight trajectory into shorter timeframe segments to reduce the computation time. Simulation results are presented to show that the proposed approach outperforms various benchmark schemes.

In order to perform the operational advantages of manned aerial vehicle (MAV) /unmanned aerial vehicle (UAV) cooperative system, a method of MAV/UAV intelligent decision-making in air combat based on deep deterministic policy gradient (DDPG) algorithm is proposed. Based on the continuous action space, four typical intentions of MAV were introduced, and the optimization models under different intentions were established. The simulation results show that the multiple UAVs can realize the cooperative operations under the guidance of MAV with a favorable effect. In addition, it makes up the gap for the 1VS1 air combat.KeywordsMAV/UAVIntention guidingDecision-makingDeep deterministic policy gradientMultiple UAVs

Mobile edge computing (MEC) networks have been recently adopted to accommodate the fast-growing number of mobile devices performing complicated tasks with limited hardware capability. Recently, edge nodes with communication, computation, and caching capacities are starting to be deployed in MEC networks. Due to the physical separation of these resources, efficient coordination and scheduling are important for efficient resource utilization and optimal network performance. In this paper, we study mobility load balancing for communication, computation, and caching-enabled heterogeneous MEC networks. Specifically, we propose to tackle this problem via a multi-agent deep reinforcement learning-based framework. Users served by overloaded edge nodes are handed over to less loaded ones, to minimize the load in the most loaded base station in the network. In this framework, the handover decision for each user is made based on the user’s own observation which comprises the user’s task at hand and the load status of the MEC network. Simulation results show that our proposed multi-agent deep reinforcement learning-based approach can reduce the time-average maximum load by up to 30% and the end-to-end delay by 50% compared to baseline algorithms.

Mobile Edge Computing (MEC) pushes computing resources from the network center to the network edge to provide efficient and reliable computing services. However, due to the mobility and diversity of mobile users (MUs), edge servers (ESs) are likely to be overloaded, which leads to a rapid decline in the quality of service provided by the MEC system. This paper introduces unmanned aerial vehicles (UAVs) to solve the problem of ES overload. This paper presents an effective Multi-UAV Collaborative Decision-Making System (MCDS) tailored to optimize task offloading and resource allocation in MEC networks. The proposed system seeks to reduce task computing delays and energy consumption while guaranteeing timely task completion and compliance with resource constraints. We design a distributed two-stage optimization algorithm to jointly optimize the task offloading decision and resource allocation of the collaborative computing system (UAVs, ESs, and MUs). To address the flight cooperation problem of multi-UAVs, we also proposed a task-aware scheduling algorithm for multi-UAVs. In addition, we conducted a large number of simulation experiments. Experimental results show that our two-stage optimization scheme is lower than other benchmark algorithms in terms of task completion delay, energy consumption and total cost of objective function. Specifically, compared with seven baseline algorithms under three different system setting scenarios, our algorithm reduces the total cost by an average of 38.79%, the energy consumption by an average of 40.70%, and the delay by an average of 15.67%.

With the rapid development of mobile edge computing (MEC) and unmanned aerial vehicle (UAV) communication networks, UAV-assisted edge computing has emerged as a promising paradigm for low-latency and energy-efficient computation. However, the time-varying nature of air-to-ground channels and the coupling between UAV trajectories and computation offloading decisions significantly increase system complexity. To address these challenges, this paper proposes an Adaptive UAV Edge Intelligence Framework (AUEIF) for joint UAV computation offloading and trajectory optimization under dynamic channels. Specifically, a dynamic graph-based system model is constructed to characterize the spatio-temporal correlation between UAV motion and channel variations. A hierarchical reinforcement learning-based optimization framework is developed, in which a high-level actor–critic module is responsible for generating coarse-grained UAV flight trajectories, while a low-level deep Q-network performs fine-grained optimization of task offloading ratios and computational resource allocation in real time. In addition, an adaptive channel prediction module leveraging long short-term memory (LSTM) networks is integrated to model temporal channel state transitions and to assist policy learning and updates. Extensive simulation results demonstrate that the proposed AUEIF achieves significant improvements in end-to-end latency, energy efficiency, and overall system stability compared with conventional deep reinforcement learning approaches and heuristic-based schemes while exhibiting strong robustness against dynamic and fluctuating wireless channel conditions.

Users in heterogeneous wireless networks may generate massive amounts of data that are delay-sensitive or require computation-intensive processing. Owing to computation ability and battery capacity limitations, wireless users (WUs) cannot easily process such data in a timely manner, and mobile edge computing (MEC) is increasingly being used to resolve this issue. Specifically, data generated by WUs can be offloaded to the MEC server for processing, which has greater computing power than WUs. However, as the location of MEC servers is fixed, unmanned aerial vehicles (UAVs) have been considered a promising solution in heterogeneous wireless networks. In this study, we design an UAV-assisted computation offloading scheme in an MEC framework with renewable power supply. The proposed model considers the instability of energy arrival, stochastic computation tasks generated by WUs, and a time-varying channel state. Owing to the complexity of the state, it is difficult to use traditional Markov decision process (MDP) with complete prior knowledge for offloading optimization. Accordingly, we propose UAV-assisted computation offloading for MEC based on deep reinforcement learning (UACODRL) to minimize the total cost, which is the weighted sum of the delay, energy consumption, and bandwidth cost. We first use the K-Means algorithm for classification to reduce the dimension of the action space. Subsequently, we use UACODRL to find the near-optimal offloading scheme to minimize the total cost. Simulations demonstrate that UACODRL converges satisfactorily and performs better than four baseline schemes with different parameter configurations.