Deep Reinforcement Learning for Computation Offloading and Resource Allocation in Blockchain-Based Multi-UAV-Enabled Mobile Edge Computing

Computation offloading services provide required computing resources for vehicles with computation-intensive tasks. Past computation offloading research mainly focused on mobile edge computing (MEC) or cloud computing, separately. This paper presents a collaborative approach based on MEC and cloud computing that offloads services to automobiles in vehicular networks. A cloud-MEC collaborative computation offloading problem is formulated through jointly optimizing computation offloading decision and computation resource allocation. Since the problem is non-convex and NP-hard, we propose a collaborative computation offloading and resource allocation optimization (CCORAO) scheme, and design a distributed computation offloading and resource allocation algorithm for CCORAO scheme that achieves the optimal solution. The simulation results show that the proposed algorithm can effectively improve the system utility and computation time, especially for the scenario where the MEC servers fail to meet demands due to insufficient computation resources.

Collaborative computation offloading and wireless charging scheduling in multi-UAV-assisted MEC networks: A TD3-based approach

This paper considers the computation offloading and resource allocation for system sum cost minimization in a mobile edge computing (MEC)network. The computation offloading decision of the user equipments (UEs)can be made through wireless link to access the MEC server. Our intent is to minimize the weighted sum cost of the system, which takes into account the energy consumption and time delay, for both task processing and data transmissions, subject to the force delay requirement of the devices. We first formulate the problem as a mixed-integer and non-convex optimization scheme. Enlightened by the superior performance of reinforcement learning (RL)on solving resource control problems, we put forward a novel RL-based scheme for computation offloading and resource allocation in MEC network. Specifically, the MEC node is considered as an “agent”, which first makes decision on whether to offloading the computing task to the MEC server, and then utilizes convex optimization algorithms to settle the radio spectrum and computation resource scheduling problem in each decision phase. The simulation results present that the RL-based method can achieve better computation offloading and resource allocation performance compared to those of local computing and remote computing modes. It also achieves very similar or slightly better performance compared to the genetic algorithm based method.

The task offloading and resource allocation strategies aim to provide efficient edge computing services for ground mobile devices in mobile edge computing (MEC) environments. In this paper, we propose a UAV-assisted dynamic deployment and decision-making framework (dDDM) to address the issue of insufficient base station coverage in remote areas or emergency scenarios, which leverages the flexibility and computational capabilities of unmanned aerial vehicle (UAV) to supplement terrestrial cellular networks. We focus on optimizing the UAV trajectory to support and enhance the effectiveness of task offloading and resource allocation strategies under conditions of user mobility and stochastic task arrivals. Therefore, a joint optimization problem is formulated with the objective of minimizing the weighted sum of the total latency of ground mobile devices and the overall energy consumption of UAV. To solve this problem, the original optimization problem is decomposed into two subproblems: UAV trajectory planning, and decision-making for task offloading & computing resource allocation. For the UAV trajectory planning subproblem, we propose an improved particle swarm optimization(PSO) algorithm integrated with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i>-means clustering, i.e., KMPSO algorithm, enabling dynamic UAV deployment in response to user mobility. For the task offloading and resource allocation decision-making, an improved black-winged kite algorithm (IBKA) is developed to determine optimal offloading and allocation strategies. Simulation results demonstrate that the proposed approach outperforms existing benchmark algorithms in terms of both latency and energy consumption.

Deep Neural Networks meet computation offloading in mobile edge networks: Applications, taxonomy, and open issues

Mobile edge computing (MEC) brings a breakthrough for Internet of Things (IoT) for its ability of offloading tasks from user equipments (UEs) to nearby servers which have rich computation resource. 5G network brings a huge breakthrough on transmission rate. Together with MEC and 5G, both execution delay of tasks and time delay from downloading would be shorter and the quality of experience (QoE) of UEs can be improved. Considering practical conditions, the computation resource of an MEC server is finite to some extent. Therefore, how to prevent the abuse of MEC resource and further allocate the resource reasonably becomes a key point for an MEC system. In this paper, an MEC system with multi-user is considered where a base station (BS) with an MEC server, which can not only provide computation offloading service but also data cache service. Especially, we take the charge for both data transmission and task computation as one part of total cost of UEs, and then explore a joint optimization for downlink resource allocation, offloading decision and computation resource allocation to minimize the total cost in terms of the time delay and the charge to UEs. The proposed problem is formulated as a mixed integer programming (MIP) one which is NP-hard. Therefore, we decouple the original problem into two subproblems which are downlink resource allocation problem and joint offloading decision and computation resource allocation problem. Then we address these two subproblems by using convex and nonconvex optimization techniques, respectively. An iterative algorithm is proposed to obtain a suboptimal solution in polynomial time. Simulation results show that our proposed algorithm performs better than benchmark algorithms.

5G beyond is an end-edge-cloud orchestrated network that can exploit heterogeneous capabilities of the end devices, edge servers, and the cloud and thus has the potential to enable computation-intensive and delay-sensitive applications via computation offloading. However, in multi user wireless networks, diverse application requirements and the possibility of various radio access modes for communication among devices make it challenging to design an optimal computation offloading scheme. In addition, having access to complete network information that includes variables such as wireless channel state, available bandwidth and computation resources, is a major issue. Deep Reinforcement Learning (DRL) is an emerging technique to address such an issue with limited and less accurate network information. In this paper, we utilize DRL to design an optimal computation offloading and resource allocation strategy for minimizing system energy consumption. We first present a multi-user end-edge-cloud orchestrated network where all devices and base stations have computation capabilities. Then, we formulate the joint computation offloading and resource allocation problem as a Markov Decision Process (MDP) and propose a new DRL algorithm to minimize system energy consumption. Numerical results based on a real-world dataset demonstrate that the proposed DRL-based algorithm significantly outperforms the benchmark policies in terms of system energy consumption. Extensive simulations show that learning rate, discount factor, and number of devices have considerable influence on the performance of the proposed algorithm.

In this paper, we investigate a new mobile blockchain-enabled edge computing (MBEC) network, where mobile users can join the empowered process of public blockchains and meanwhile offload computation-intensive mining tasks to the mobile edge computing (MEC) server. However, the trustiness of the MEC server and the fairness of computation resources allocated by the MEC server for each user become key challenges. To tackle these challenges, we consider an untrusted MEC server and propose a nonce hash computing ordering (HCO) mechanism in MBEC networks. Then we formulate nonce hash computing demands of an individual user as a non-cooperative game that maximizes the personal revenue. Moreover, we also analyze the existence of Nash equilibrium of the non-cooperative game and design an alternating optimization algorithm to achieve the optimal nonce selection strategies for all users. With the proposed HCO mechanism, the MEC server can provide much fairer computation resources for all users, and we can achieve the optimal nonce strategies of hash computing demands by using the proposed alternating optimization algorithm. Numerical results demonstrate that the proposed HCO mechanism can provide fairer computation resource allocation than the traditional weighted round-robin mechanism, and further verify the effectiveness of this alternating optimization algorithm.

In the future mobile edge networks, the Internet of things (IoT) applications will be latency-sensitive and computationally intensive. Given the resource limitation of IoT devices, mobile edge computing (MEC) servers are critical to support the efficient processing of IoT tasks. Since MEC servers attached to the ground base stations are generally deployed in fixed locations and vulnerable to physical damage, the unmanned aerial vehicle (UAV) and satellite-assisted MEC framework has been proposed to leverage the flexibility of UAVs and the broad coverage of satellites. However, efficient utilization of the UAV/satellite resources is challenging for the static ground IoT devices because of the dynamic in terms of aerial and space network topology and IoT task arrival rates. To adapt to the changing environment and utilize the interaction among multiple UAVs, we propose a multi-agent deep deterministic policy gradient (MADDPG) framework to jointly optimize the traveling routes of multi-UAVs and the offloading decision of IoT devices. To minimize the processing cost in terms of task processing latency and energy consumption of IoT devices, cooperative UAVs can help find the optimal task offloading location for each IoT device. Simulation results show the proposed algorithm based on MADDPG can averagely decrease 20% of the above processing cost compared with the benchmark approach.

Sparse code multiple access (SCMA) is a kind of code-domain non-orthogonal multiple access (NOMA) scheme, which can support the increasing requirements for high spectral efficiency and massive connections. Meanwhile, multi-access edge computing (MEC) is a promising technology for providing resource-constrained users with computing resources. The integration of these two technologies can efficiently improve the computation service. In this paper, we propose a novel optimization scheme in the SCMA-based MEC network from the perspective of energy and latency for the Internet of Things (IoT) devices. To minimize the total overhead of devices, the communication and computation resources allocation, as well as computation offloading are jointly considered. Primarily, a joint SCMA codebook assignment based on the bidirectional matching principle and optimal power allocation algorithm is proposed to maximize the uplink transmit rate. Moreover, we put forward CPU frequency scheduling strategies utilizing convex optimization to optimize the computing resources allocation of local devices and the MEC server. Finally, a low-complexity task offloading policy based on simulated annealing is presented. Numerical results show that our proposed joint optimization algorithm for resource allocation and computation offloading can achieve a good compromise between time delay and energy consumption for IoT devices.

With the ongoing advancement of electric power Internet of Things (IoT), traditional power inspection methods face challenges such as low efficiency and high risk. Unmanned aerial vehicles (UAVs) have emerged as a more efficient solution for inspecting power facilities due to their high maneuverability, excellent line-of-sight communication capabilities, and strong adaptability. However, UAVs typically grapple with limited computational power and energy resources, which constrain their effectiveness in handling computationally intensive and latency-sensitive inspection tasks. In response to this issue, we propose a UAV task offloading strategy based on deep reinforcement learning (DRL), which is designed for power inspection scenarios consisting of mobile edge computing (MEC) servers and multiple UAVs. Firstly, we propose an innovative UAV-Edge server collaborative computing architecture to fully exploit the mobility of UAVs and the high-performance computing capabilities of MEC servers. Secondly, we established a computational model concerning energy consumption and task processing latency in the UAV power inspection system, enhancing our understanding of the trade-offs involved in UAV offloading strategies. Finally, we formalize the task offloading problem as a multi-objective optimization issue and simultaneously model it as a Markov Decision Process (MDP). Subsequently, we proposed a task offloading algorithm based on a Deep Deterministic Policy Gradient (OTDDPG) to obtain the optimal task offloading strategy for UAVs. The simulation results demonstrated that this approach outperforms baseline methods with significant improvements in task processing latency and energy consumption.

Mobile Edge Computing (MEC) migrates the computing center to the network edge to provide computing services for Mobile Users (MUs). However, due to the limited capacity of MUs and insufficient types of services stored by edge servers, it brings great challenges to computation offloading and service caching in Internet of Things (IoT) network. In this paper, we investigate the joint optimization problem of computation offloading, resource allocation and service caching replacement in a collaborative MEC system to minimize the total cost of all MUs under the latency and resource constraints. Accordingly, we formulate the problem as a Mixed Integer Non-linear Programming (MINLP) problem. In order to solve this problem, the Deep Deterministic Policy Gradient (DDPG)-based algorithm is proposed. The simulation results show that the proposed algorithm is better than other benchmark schemes in different scenarios.

Mobile edge computing (MEC)-enabled satellite–terrestrial networks (STNs) can provide task computing services for Internet of Things (IoT) devices. However, since some applications’ tasks require huge amounts of computing resources, sometimes the computing resources of a local satellite’s MEC server are insufficient, but the computing resources of neighboring satellites’ MEC servers are redundant. Therefore, we investigated inter-satellite cooperation in MEC-enabled STNs. First, we designed a system model of the MEC-enabled STN architecture, where the local satellite and the neighboring satellites assist IoT devices in computing tasks through inter-satellite cooperation. The local satellite migrates some tasks to the neighboring satellites to utilize their idle resources. Next, the task completion delay minimization problem for all IoT devices is formulated and decomposed. Then, we propose an inter-satellite cooperative joint offloading decision and resource allocation optimization scheme, which consists of a task offloading decision algorithm based on the Grey Wolf Optimizer (GWO) algorithm and a computing resource allocation algorithm based on the Lagrange multiplier method. The optimal solution is obtained by continuous iterations. Finally, simulation results demonstrate that the proposed scheme achieves relatively better performance than other baseline schemes.

Resource allocation in RISs-assisted UAV-enabled MEC network with computation capacity improvement

With the emergence of various new Internet of Things (IoT) devices and the rapid increase in the number of users, enormous services and complex applications are growing rapidly. However, these services and applications are resource-intensive and data-hungry, requiring satisfactory quality-of-service (QoS) and network coverage density guarantees in sparsely populated areas, whereas the limited battery life and computing resources of IoT devices will inevitably become insufficient. Unmanned aerial vehicle (UAV)-enabled mobile edge computing (MEC) is one of the most promising solutions that ensures the stability and expansion of the network coverage area for these applications and provides them with computational capabilities. In this paper, computation offloading and resource allocation are jointly considered for multi-user multi-UAV-enabled mobile edge-cloud computing systems. First, we propose an efficient resource allocation and computation offloading model for a multi-user multi-UAV-enabled mobile edge-cloud computing system. Our proposed system is scalable and can support increases in network traffic without performance degradation. In addition, the network deploys multi-level mobile edge computing (MEC) technology to provide the computational capabilities at the edge of the radio access network (RAN). The core network is based on software-defined networking (SDN) technology to manage network traffic. Experimental results demonstrate that the proposed model can dramatically boost the system performance of the system in terms of time and energy.