Deep Reinforcement Learning-Based Resource Management for UAV-Assisted Mobile Edge Computing Against Jamming

In this paper, we propose a secure unmanned aerial vehicle (UAV) mobile edge computing (MEC) system where multiple ground users offload large computing tasks to a nearby legitimate UAV in the presence of multiple eavesdropping UAVs with imperfect locations. To enhance security, jamming signals are transmitted from both the full-duplex legitimate UAV and non-offloading ground users. For this system, we design a low-complexity iterative algorithm to maximize the minimum secrecy capacity subject to latency, minimum offloading and total power constraints. Specifically, we jointly optimize the UAV location, users' transmit power, UAV jamming power, offloading ratio, UAV computing capacity, and offloading user association. Numerical results show that our proposed algorithm significantly outperforms baseline strategies over a wide range of UAV self-interference (SI) efficiencies, locations and packet sizes of ground users. Furthermore, we show that there exists a fundamental tradeoff between the security and latency of UAV-enabled MEC systems which depends on the UAV SI efficiency and total UAV power constraints.

In recent years, low-latency tasks such as real-time communication, virtual reality, and augmented reality have higher latency requirements. Since users have relatively weak computing power, the past practice was to offload low-latency tasks to base stations (BSs) with more powerful computing power for execution, which can reduce computing latency in real-time communication. Duetothe inflexible deployment of BSs and the uneven distribution of computing resources, users often find it difficult to meet the needs of low-latency tasks in places that lack BSs, so unmanned aerial vehicles (UAVs) are needed to alleviate this problem. UAVs can serve as communication relays between users and BSs and provide computing services directly to users. However, when there are few BSs, UAVs may experience load imbalance, that is, some UAVs are overloaded while other UAVs are idle. To solve this problem, we propose a UAV collaborative three-layer mobile edge computing (MEC) system model and describe the problem as a mixed integer nonlinear programming (MINLP) problem. Then, we propose an extended genetic algorithm for UAV collaboration in three-layer MEC task allocation and optimization (EGAC-3MEC) where the three-layer structure includes users, UAVs, and BSs. The proposed EGAC- 3MEC achieves reasonable allocation of tasks and resources through offloading and resource allocation decisions, reduces system delay and energy consumption as much as possible, and prioritizes the delay of high-priority users. Experimental results show that the algorithm in this paper converges quickly when various parameters change, achieves better performance than the baseline algorithms when the user task size is different, and moreover, the necessity of UAV collaboration is proven.

Unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) system is a prominent strategy where a UAV equipped with an MEC server is deployed to serve terminal devices. This paper considers a multi-UAV assisted network in which multiple UAVs and a terrestrial base station (BS) are deployed to provide MEC services to mobile users. The objective is to minimize an energy and latency-based cost function by jointly optimizing task offloading and MEC server selection decision, transmission power, UAV trajectory, and CPU frequency allocation. An alternating iterative approach based on the block descent method is proposed to solve this problem. In the first layer, task offloading and server selection decision subproblem is solved using a game theoretic approach. The second layer handles offloading and downloading transmission power allocations by utilizing a simplistic geometric waterfilling (GWF) technique, and the UAV trajectory by successive convex approximation (SCA). Whereas, the third layer solves the computation resource subproblem by performing CPU frequency allocation using a gradient descent method. The proposed method uses a segment-by-segment approach, which divides the entire UAV flight trajectory into shorter timeframe segments to reduce the computation time. Simulation results are presented to show that the proposed approach outperforms various benchmark schemes.

It is essential to locate the victims timely for efficient rescue after the disaster occurs in the global positioning system (GPS) denied forest area due to the influence of the tree shading. Existing works have studied optimizing the trajectory of the unmanned aerial vehicle (UAV) to exploit the wireless signal from the ground users for localization. However, current works mainly focus on optimizing the localization accuracy while paying limited attention to the localization task completion time, which makes it challenging to meet the timeliness requirements of emergency rescue missions. To provide accurate localization services for the ground victims quickly, we propose a multi-agent deep reinforcement learning (MA-DRL)-based UAV trajectory planning algorithm, which can provide high-efficiency cooperation between the multiple UAVs by exploiting the prior information and measurements from the partners. Specifically, a low-resolution trajectory planning algorithm is proposed to reduce redundant flight distances in the pre-fly stage to localize the victim’s quantity and dispersion. Furthermore, to provide high localization accuracy and energy-efficiency victims location services quickly, we exploit the coarse user information from the pre-fly stage, integrate an adaptive forest channel model and UAV energy consumption model, and propose an MA-DRL-based UAV trajectory planning algorithm which can perform decentralized execution for the high-efficiency cooperation localization. Simulation results show that our method can finish localization missions faster with less energy consumption while guaranteeing the localization accuracy compared to other benchmark algorithms.

In current world, processing the data by the users, researchers, organizations, etc. through online source (ie. cloud) is increasing tremendously, where it offers various services like storing the data, allocating the resources, retrieval of data, security, etc. Since the data is stored in cloud, processing it leads to increase in latency, due to its long-distance communication. As distance increases, data transmission produces a delay between source and destination which is called as latency. To get over this problem, MEC (Mobile Edge Computing) system can be positioned between the cloud infrastructure and the user equipment(UE) in the IoT environment. Instead of sending data to the cloud directly which is collected by IoT sensors, edge computing processes those data within the network. The study of this review focuses on the various techniques such as deep learning algorithms, deep reinforcement learning approaches for offloading decisions and optimization techniques in 5G networks for minimizing the latency in the MEC systems, for producing higher processing capability to the clients. For increasing the MEC system performance, efficient offloading strategies can be used to minimize latency for various tasks.

This article establishes a new multiunmanned aerial vehicle (multi-UAV)-enabled mobile edge computing (MEC) system, where a number of unmanned aerial vehicles (UAVs) are deployed as flying edge clouds for large-scale mobile users. In this system, we need to optimize the deployment of UAVs, by considering their number and locations. At the same time, to provide good services for all mobile users, it is necessary to optimize task scheduling. Specifically, for each mobile user, we need to determine whether its task is executed locally or on a UAV (i.e., offloading decision), and how many resources should be allocated (i.e., resource allocation). This article presents a two-layer optimization method for jointly optimizing the deployment of UAVs and task scheduling, with the aim of minimizing system energy consumption. By analyzing this system, we obtain the following property: the number of UAVs should be as small as possible under the condition that all tasks can be completed. Based on this property, in the upper layer, we propose a differential evolution algorithm with an elimination operator to optimize the deployment of UAVs, in which each individual represents a UAV's location and the entire population represents an entire deployment of UAVs. During the evolution, we first determine the maximum number of UAVs. Subsequently, the elimination operator gradually reduces the number of UAVs until at least one task cannot be executed under delay constraints. This process achieves an adaptive adjustment of the number of UAVs. In the lower layer, based on the given deployment of UAVs, we transform the task scheduling into a 0-1 integer programming problem. Due to the large-scale characteristic of this 0-1 integer programming problem, we propose an efficient greedy algorithm to obtain the near-optimal solution with much less time. The effectiveness of the proposed two-layer optimization method and the established multi-UAV-enabled MEC system is demonstrated on ten instances with up to 1000 mobile users.

Mobile Edge Computing (MEC) is a promising computing paradigm, which can provide mobile devices with powerful computing capability and low delay by offloading resource-intensive tasks to the MEC Server. However, due to the varying wireless channel conditions and different task delay requirements, the resource allocation decision made by MEC Server cannot always achieve the lowest system energy consumption. In this paper, we consider resource allocation problem in a multi-user MEC system. The objective is to achieve minimum system energy consumption under delay constraint by suitably allocating communication and computing resources. We solve the communication resources (subcarrier) allocation problem through regression algorithm, make use of three regression algorithms to predict subcarrier allocation. And then calculate the computing resources (CPU frequency) allocation and system energy consumption. Simulation results show that machine learning can achieve near-optimal system energy consumption, where the system energy consumption of the ANN(Artificial Neural Network), Xgboost (Extreme Gradient Boosting), and RF(Random Forest) algorithms is 1.30%, 7.49%, and 23.29% higher than the optimal system energy consumption respectively. In the view of prediction loss, ANN and Xgboost perform better than RF.

Mobile Edge Computing (MEC) pushes computing resources from the network center to the network edge to provide efficient and reliable computing services. However, due to the mobility and diversity of mobile users (MUs), edge servers (ESs) are likely to be overloaded, which leads to a rapid decline in the quality of service provided by the MEC system. This paper introduces unmanned aerial vehicles (UAVs) to solve the problem of ES overload. This paper presents an effective Multi-UAV Collaborative Decision-Making System (MCDS) tailored to optimize task offloading and resource allocation in MEC networks. The proposed system seeks to reduce task computing delays and energy consumption while guaranteeing timely task completion and compliance with resource constraints. We design a distributed two-stage optimization algorithm to jointly optimize the task offloading decision and resource allocation of the collaborative computing system (UAVs, ESs, and MUs). To address the flight cooperation problem of multi-UAVs, we also proposed a task-aware scheduling algorithm for multi-UAVs. In addition, we conducted a large number of simulation experiments. Experimental results show that our two-stage optimization scheme is lower than other benchmark algorithms in terms of task completion delay, energy consumption and total cost of objective function. Specifically, compared with seven baseline algorithms under three different system setting scenarios, our algorithm reduces the total cost by an average of 38.79%, the energy consumption by an average of 40.70%, and the delay by an average of 15.67%.

Unmanned Aerial Vehicle (UAV)-enabled wirelessly powered Mobile Edge Computing (MEC) is emerging as a powerful technology for boosting computational capability and energy supplementation in Internet of Things (IoT). This work addresses the long-term task completion ratio maximization problem in UAV-enabled wirelessly powered MEC systems. Besides the large number of optimization parameters, the environment can only be partially observed as the UAVs cannot cover the whole network area. Then, it is very challenging to obtain good solutions due to the lack of global information. We introduce a novel distributed Multi-Agent Deep Reinforcement Learning (MADRL) framework for optimizing UAVs’ actions and resource allocation, considering the constraints of tasks that vary in size, arrival times, and required computation completion time. To decouple the complicated parameters, we divide the problem into two manageable subproblems—UAVs’ action decision and resource allocation under a given UAV’s action. We employ a distributed Deep Reinforcement Learning (DRL) scheme for the former subproblem to cope with the partially observable nature. By revealing some important properties of the later subproblem, we design an efficient two-stage optimal algorithm to minimize the total consumed energy of nodes while maximizing the task-completing number. Extensive simulations validate the effectiveness of the proposed framework, achieving over a 50% improvement in task completion ratio compared to baseline schemes in some scenarios.

This article presents a cooperative multiagent deep reinforcement learning (MADRL) approach for unmanned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks. An UAV with computing capability can provide task offlaoding services to ground Internet of Things devices (IDs). With partial observation of the entire network state, the UAV and the IDs individually determine their MEC strategies, i.e., UAV trajectory, resource allocation, and task offloading policy. This requires joint optimization of decision-making process and coordination strategies among the UAV and the IDs. To address this difficulty, the proposed cooperative MADRL approach computes two types of action variables, namely, message action and solution action, each of which is generated by dedicated actor neural networks (NNs). As a result, each agent can automatically encapsulate its coordination messages to enhance the MEC performance in the decentralized manner. The proposed actor structure is designed based on graph attention networks such that operations are possible regardless of the number of IDs. A scalable training algorithm is also proposed to train a group of NNs for arbitrary network configurations. Numerical results demonstrate the superiority of the proposed cooperative MADRL approach over conventional methods.

Collaborative computation offloading and wireless charging scheduling in multi-UAV-assisted MEC networks: A TD3-based approach

Driven by the exploding computing service demands from various intelligent mobile applications, an increasing amount of research efforts have been devoted to mobile edge computing (MEC). Meanwhile, unmanned aerial vehicles (UAVs) have found a great success in assisting existing wireless systems due to their flexibility and low cost. The advancements in these two closely related fields foster the development of the recently advocated UAV-assisted MEC paradigm, which is expected to bring unprecedented performance gain to the existing ground-based MEC systems. Nonetheless, existing works on UAV-assisted MEC mainly focus on the single-UAV scenarios and often assume static system states. In this paper, UAV swarm assisted MEC is considered where multiple collaborative UAVs are employed to help the terrestrial edge server to provide better edge computing services. However, the complexity of using existing methods to find the best dynamic coordination strategy in UAV swarm assisted MEC becomes intractable when the number of UAVs increases. To resolve this challenge, a novel decentralized deep reinforcement learning algorithm is proposed in this work, which can reduce the complexity by orders of magnitude. In addition, simulations are conducted to show that by using the proposed algorithm, the UAV swarm can efficiently learn a good dynamic coordination strategy and thus achieve a significantly better performance than the baseline scheme.

Mobile edge computing (MEC) powered by unmanned aerial vehicles (UAVs), with the advantages of flexible deployment and wide coverage, is a promising technology to solve computationally intensive communication problems. In this paper, an orthogonal time frequency space (OTFS)-based UAV-assisted MEC system is studied, in which OTFS technology is used to mitigate the Doppler effect in UAV high-speed mobile communication. The weighted total energy consumption of the system is minimized by jointly optimizing the time division, CPU frequency allocation, transmit power allocation and flight trajectory while considering Doppler compensation. Thus, the resultant problem is a challenging nonconvex problem. We propose a joint algorithm that combines the benefits of the atomic orbital search (AOS) algorithm and convex optimization. Firstly, an improved AOS algorithm is proposed to swiftly obtain the time slot allocation and high-quality solution of the UAV optimal path. Secondly, the optimal solution for the CPU frequency and transmit power allocation is found by using Lagrangian duality and the first-order Taylor formula. Finally, the optimal solution of the original problem is iteratively obtained. The simulation results show that the weighted total energy consumption of the OTFS-based system decreases by 13.6% compared with the orthogonal frequency division multiplexing (OFDM)-based system. The weighted total energy consumption of the proposed algorithm decreases by 11.7% and 26.7% compared with convex optimization and heuristic algorithms, respectively.

Mobile edge computing (MEC) has emerged as a key enabler for low-latency and computation-intensive applications. However, conventional terrestrial MEC networks face challenges in coverage, scalability, and dynamic environments. To address these limitations, this paper explores unmanned aerial vehicle (UAV)-assisted MEC, leveraging the mobility and flexibility of UAVs to enhance edge computing performance. However, challenges such as dynamic mobile user association, multi-UAV collaborative trajectory design, and limited communication and computation resources remain significant hurdles. We then formulate a joint optimization problem for user association, trajectory design, and resource allocation for the multi-UAV collaborative MEC network, aiming to minimize system cost considering computation task demands and UAV resource constraints. The problem, modeled as a Mixed Integer NonLinear Program (MINLP), is reformulated as a decentralized partially observable Markov decision process (Dec-POMDP) to capture complex system dynamics. Afterwards, we propose a multi-agent generative-enhanced deep reinforcement learning method that integrates generative diffusion model (GDM) with multi-agent deep reinforcement learning (MADRL) to effectively handle high-dimensional action spaces and enable collaborative multi-UAV decision-making. Extensive simulations using a real-world mobility dataset demonstrate that the proposed method consistently outperforms three baselines in terms of time-average system cost, task completion delay, task satisfactory ratio, and energy consumption ratio when varying the number of UAVs.

The unmanned aerial vehicle (UAV) enabled mobile edge computing (MEC) is a promising paradigm for providing extensive coverage and additional computation services to the Internet of Things (IoT) devices with smart applications, which are normally computation-intensive and delay-critical, especially in some remote areas or emerging scenarios without available infrastructure. However, considering the restricted computation capabilities of UAVs, even multiple UAVs may not satisfy the quality of service (QoS) requirement of IoT users with heavy-computation applications. In this paper, a heterogenous-multiple-UAVs enabled collaborative aerial-ground MEC framework is proposed. The total computation bits maximization problem under the time-division multiple access (TDMA) scheme and partial offloading mode is formulated by jointly optimizing the user association, the CPU-cycle frequencies of users and computing UAVs, the transmit power of users and UAVs, the offloading time of users and UAVs, as well as the UAVs' trajectories. Due to the non-convexity of the original problem with the nonlinear coupling of different variables, a two-layered alternative optimization (TLAO) algorithm is proposed, where the outer layer optimizes the user association and resource allocation, while the inner layer optimizes the UAV trajectory scheduling. Simulation results demonstrate that our proposed TLAO scheme can improve the total computation bits compared with other benchmark schemes, and achieve fast convergence. Moreover, comparative results also verify that our proposed strategy is flexible to apply to homogeneous multi-UAVs and single-UAV scenarios.