Cooperative Multiagent Deep Reinforcement Learning Methods for UAV-Aided Mobile Edge Computing Networks

This chapter studies mobile edge computing (MEC) networks assisted by unmanned aerial vehicles (UAVs). According to the application scenarios, we consider three roles for UAVs in MEC networks: exploiting MEC computing capabilities, serving as a computing server, and serving as a relay for computation offloading. Furthermore, the details for resource allocation and optimization are presented in the three scenarios of UAV-assisted MEC networks. In addition, we focus on the situation in which a UAV not only functions as an MEC server to inspect turbines on a wind farm, but also performs task computation. To facilitate wide applications of UAV-assisted MEC in practice, this chapter highlights the main implementation issues of UAV-assisted MEC, including optimal UAV deployment, wind models, and joint trajectory–computation performance optimization.

Collaborative computation offloading and wireless charging scheduling in multi-UAV-assisted MEC networks: A TD3-based approach

With the rapid development of mobile edge computing (MEC) and unmanned aerial vehicle (UAV) communication networks, UAV-assisted edge computing has emerged as a promising paradigm for low-latency and energy-efficient computation. However, the time-varying nature of air-to-ground channels and the coupling between UAV trajectories and computation offloading decisions significantly increase system complexity. To address these challenges, this paper proposes an Adaptive UAV Edge Intelligence Framework (AUEIF) for joint UAV computation offloading and trajectory optimization under dynamic channels. Specifically, a dynamic graph-based system model is constructed to characterize the spatio-temporal correlation between UAV motion and channel variations. A hierarchical reinforcement learning-based optimization framework is developed, in which a high-level actor–critic module is responsible for generating coarse-grained UAV flight trajectories, while a low-level deep Q-network performs fine-grained optimization of task offloading ratios and computational resource allocation in real time. In addition, an adaptive channel prediction module leveraging long short-term memory (LSTM) networks is integrated to model temporal channel state transitions and to assist policy learning and updates. Extensive simulation results demonstrate that the proposed AUEIF achieves significant improvements in end-to-end latency, energy efficiency, and overall system stability compared with conventional deep reinforcement learning approaches and heuristic-based schemes while exhibiting strong robustness against dynamic and fluctuating wireless channel conditions.

Resource allocation in RISs-assisted UAV-enabled MEC network with computation capacity improvement

Unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) system is a prominent strategy where a UAV equipped with an MEC server is deployed to serve terminal devices. This paper considers a multi-UAV assisted network in which multiple UAVs and a terrestrial base station (BS) are deployed to provide MEC services to mobile users. The objective is to minimize an energy and latency-based cost function by jointly optimizing task offloading and MEC server selection decision, transmission power, UAV trajectory, and CPU frequency allocation. An alternating iterative approach based on the block descent method is proposed to solve this problem. In the first layer, task offloading and server selection decision subproblem is solved using a game theoretic approach. The second layer handles offloading and downloading transmission power allocations by utilizing a simplistic geometric waterfilling (GWF) technique, and the UAV trajectory by successive convex approximation (SCA). Whereas, the third layer solves the computation resource subproblem by performing CPU frequency allocation using a gradient descent method. The proposed method uses a segment-by-segment approach, which divides the entire UAV flight trajectory into shorter timeframe segments to reduce the computation time. Simulation results are presented to show that the proposed approach outperforms various benchmark schemes.

Using Unmanned aerial vehicles (UAVs) to assist mobile edge computing (MEC) network is a promising solution to provide flexible and low-latency computing service for resource limited user equipments (UEs). However, the computing resources that one UAV can provide are often limited and cannot provide computing services for massive mobile users, thus limiting the application of this technology. So, a novel multi-UAV-assisted MEC network is proposed in this paper. The optimization problem is formulated with the aim to make full use of network's computing resources to reduce the number of failed tasks and energy consumption. To solve this challenging problem, a novel solution is proposed. First, we use a swarm intelligence algorithm to investigate the deployment of UAVs. Second, we offer an efficient matching algorithm to explore the tasks assignments under the given deployment of UAVs. The simulation results show that the proposed solution can effectively reduce the number of failed tasks and energy consumption.

Mobile edge computing (MEC) has emerged as a key enabler for low-latency and computation-intensive applications. However, conventional terrestrial MEC networks face challenges in coverage, scalability, and dynamic environments. To address these limitations, this paper explores unmanned aerial vehicle (UAV)-assisted MEC, leveraging the mobility and flexibility of UAVs to enhance edge computing performance. However, challenges such as dynamic mobile user association, multi-UAV collaborative trajectory design, and limited communication and computation resources remain significant hurdles. We then formulate a joint optimization problem for user association, trajectory design, and resource allocation for the multi-UAV collaborative MEC network, aiming to minimize system cost considering computation task demands and UAV resource constraints. The problem, modeled as a Mixed Integer NonLinear Program (MINLP), is reformulated as a decentralized partially observable Markov decision process (Dec-POMDP) to capture complex system dynamics. Afterwards, we propose a multi-agent generative-enhanced deep reinforcement learning method that integrates generative diffusion model (GDM) with multi-agent deep reinforcement learning (MADRL) to effectively handle high-dimensional action spaces and enable collaborative multi-UAV decision-making. Extensive simulations using a real-world mobility dataset demonstrate that the proposed method consistently outperforms three baselines in terms of time-average system cost, task completion delay, task satisfactory ratio, and energy consumption ratio when varying the number of UAVs.

This paper considers the computation offloading and resource allocation for system sum cost minimization in a mobile edge computing (MEC)network. The computation offloading decision of the user equipments (UEs)can be made through wireless link to access the MEC server. Our intent is to minimize the weighted sum cost of the system, which takes into account the energy consumption and time delay, for both task processing and data transmissions, subject to the force delay requirement of the devices. We first formulate the problem as a mixed-integer and non-convex optimization scheme. Enlightened by the superior performance of reinforcement learning (RL)on solving resource control problems, we put forward a novel RL-based scheme for computation offloading and resource allocation in MEC network. Specifically, the MEC node is considered as an “agent”, which first makes decision on whether to offloading the computing task to the MEC server, and then utilizes convex optimization algorithms to settle the radio spectrum and computation resource scheduling problem in each decision phase. The simulation results present that the RL-based method can achieve better computation offloading and resource allocation performance compared to those of local computing and remote computing modes. It also achieves very similar or slightly better performance compared to the genetic algorithm based method.

As maritime activities become more frequent, many computationally intensive applications have emerged. Unmanned aerial vehicles (UAVs) can be utilized to provide computing services for maritime networks. In this paper, a multi-UAV assisted mobile edge computing (MEC) network is designed. Each UAV is configured with a nano-server to provide computational offloading services for maritime users (MVs). The goal is to minimize the maximum processing delay of the multi-VAV assisted MEC network. We present an optimization problem for joint task offloading, resource allocation, and flight trajectory of UAVs. The above problem is an mixed non-integer linear programming problem (MINLP), which is transformed into a Markov decision process (MDP). A computational offload algorithm based on deep deterministic policy gradient (DDPG) is proposed to solve this problem. Simulation results reveal that the DDPG algorithm can achieve fast convergence and minimize processing delay compared with baseline algorithms.

Mobile edge computing (MEC) networks have been recently adopted to accommodate the fast-growing number of mobile devices performing complicated tasks with limited hardware capability. Recently, edge nodes with communication, computation, and caching capacities are starting to be deployed in MEC networks. Due to the physical separation of these resources, efficient coordination and scheduling are important for efficient resource utilization and optimal network performance. In this paper, we study mobility load balancing for communication, computation, and caching-enabled heterogeneous MEC networks. Specifically, we propose to tackle this problem via a multi-agent deep reinforcement learning-based framework. Users served by overloaded edge nodes are handed over to less loaded ones, to minimize the load in the most loaded base station in the network. In this framework, the handover decision for each user is made based on the user’s own observation which comprises the user’s task at hand and the load status of the MEC network. Simulation results show that our proposed multi-agent deep reinforcement learning-based approach can reduce the time-average maximum load by up to 30% and the end-to-end delay by 50% compared to baseline algorithms.

To address the severe challenge from the rapid growth of number of downloads of contents via mobile network, it is imperative to develop efficient content caching scheme in mobile edge computing (MEC) networks, which can alleviate the heavy burden on backhaul links and reduce the delay for content delivery. However, the caching performance is highly related to the MEC server placement. In this paper, we jointly optimize the server placement problem and content caching problem in MEC networks. We propose a heuristic algorithm based on Kuhn-Munkres (KM) algorithm and greedy algorithm to minimize the average delay and average bandwidth resource usage. The simulation shows that our proposed algorithm can reduce delay and bandwidth, compared with other schemes.

Re-establishing emergency communication and ensuring rapid response are critical for rescue operations in natural disaster scenarios, such as earthquakes, floods, and wildfires. Unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) networks have emerged as a promising solution to re-establish communication links and provide flexible computational support in these complex environments. However, the existing UAV-assisted MEC research has not fully investigated the joint optimization of task offloading and resource allocation (JTORA) problem, considering both terrain obstacles and high-interference zones. In this paper, we investigate the JTORA problem to minimize the energy consumption for communication and computation in a rate-splitting multiple access (RSMA)-based UAV-assisted mobile edge computing (MEC) network. RSMA is utilized to enhance interference management and improve spectral efficiency. We propose a proximal policy optimization (PPO)-based method to optimize the task offloading ratio, message splitting ratio, and RSMA precoding matrix for the proposed JTORA problem. Simulation results show that the proposed approach effectively enhances system efficiency and sustainability.

Energy-efficient communication is one of the most pertaining issues in realizing mobile edge computing (MEC) networks. How to effectively use non-orthogonal multiple access (NOMA) to realize energy efficient MEC networks has been of high interest and calls for in-depth study. In particular, there has been yet no work to study the energy efficient communication scheme from system perspective in this area. In this work, we focus on the design of a system-centric computation energy efficiency (EE) scheme in a NOMA-based MEC network. The system computation EE is maximized by jointly optimizing the computing frequencies of the MEC server and the local users, the offloading time, the transmit powers of local users, as well as the execution time for the local users and the MEC server. Employing the fractional programming theory and introducing auxiliary variables, we develop an efficient Dinkelbach-based iterative scheme to tackle the formulated non-linear fractional programming problem. For obtaining instrumental insights, we further derive closed-form expressions for the partial solutions based on Karush-Kuhn-Tucker conditions. Simulation results are provided to verify the quick convergence of the proposed iterative algorithm, to obtain insights, as well as to demonstrate the superiority of the proposed resource allocation scheme in terms of the system computation EE by comparing it with benchmark schemes.

With the growth of data scale in the mobile edge computing (MEC) network, data security of the MEC network has become a burning concern. The application of blockchain technology in MEC enhances data security and privacy protection. However, throughput becomes the bottleneck of the blockchain-enabled MEC system. Hence, this paper proposes a novel hierarchical and partitioned blockchain framework to improve scalability while guaranteeing the security of partitions. Next, we model the joint optimization of throughput and security as a Markov decision process (MDP). After that, we adopt deep reinforcement learning (DRL) based algorithms to obtain the number of partitions, the size of micro blocks and the large block generation interval. Finally, we analyze the security and throughput performance of proposed schemes. Simulation results demonstrate that proposed schemes can improve throughput while ensuring the security of partitions.

The stochastic geometry modeling and analysis of mobile edge computing (MEC) networks contribute to the effective network configuration. Most prior works investigate the MEC network performance by spatial averaging the topological randomness, which can hide the differences among users and result in inaccurate analytical results. To address this limitation, an analytical framework is developed to study the meta distribution of the task offloading performance. We consider the Poisson MEC network where the users with dynamic task requests offload the tasks to the MEC server. Given the model, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$b$</tex-math></inline-formula> -th moment and the meta distribution of the success task offloading probability are first analyzed. Further, the exact yet convenient approximation of the MEC computing load is derived, based on which the edge computing delay is obtained. Our results demonstrate that the proposed meta-analytical framework allows tractable yet accurate analytical results, and provides additional fine-grained information about MEC-network planning.