Novel method of mobile edge computation offloading based on evolutionary game strategy for IoT devices

In the current fifth-generation (5G) and Beyond 5G (B5G) era, the Unmanned Aerial Vehicles (UAVs) have been playing a vital role and attracting interest in different application areas in the military, and civil applications such as communications, disaster management, search and rescue, security, control, agriculture, Internet of things (IoT), etc. In these networks, ultra-heterogeneous IoT devices generate time-sensitive traffic. However, those devices have limited resources to compute tasks. Recently, Mobile Edge Computation Offloading (MECO) has been considered as an encouraging model to enable the computation tasks of IoT devices to be performed by MEC servers and support ultra-low latency IoT applications to ensure Quality of services (QoS). However, terrestrial network failure due to natural and human-made disasters has been increasing, and difficult to provide reliable computation offloading and resource allocation services to IoT networks. Nowadays, UAVs have been promising technology to quickly deploy and recover the system to provide efficient services to edge nodes. The offloading and resource allocation problems in current network technology are complex, and offloading task to edge server is vulnerable to security risks. Hence, we utilize a deep reinforcement learning method to handle a complex problem for computation offloading and resource allocations in a dynamic environment. And also, we explore a blockchain-based multi-UAV-assisted MEC architecture in securing and optimizing the offloading problems.

Internet of Things (IoT) devices can apply mobile edge computing (MEC) and energy harvesting (EH) to provide high-level experiences for computational intensive applications and concurrently to prolong the lifetime of the battery. In this paper, we propose a reinforcement learning (RL) based offloading scheme for an IoT device with EH to select the edge device and the offloading rate according to the current battery level, the previous radio transmission rate to each edge device, and the predicted amount of the harvested energy. This scheme enables the IoT device to optimize the offloading policy without knowledge of the MEC model, the energy consumption model, and the computation latency model. Further, we present a deep RL-based offloading scheme to further accelerate the learning speed. Their performance bounds in terms of the energy consumption, computation latency, and utility are provided for three typical offloading scenarios and verified via simulations for an IoT device that uses wireless power transfer for energy harvesting. Simulation results show that the proposed RL-based offloading scheme reduces the energy consumption, computation latency, and task drop rate, and thus increases the utility of the IoT device in the dynamic MEC in comparison with the benchmark offloading schemes.

Mobile Edge Computing (MEC) migrates the computing center to the network edge to provide computing services for Mobile Users (MUs). However, due to the limited capacity of MUs and insufficient types of services stored by edge servers, it brings great challenges to computation offloading and service caching in Internet of Things (IoT) network. In this paper, we investigate the joint optimization problem of computation offloading, resource allocation and service caching replacement in a collaborative MEC system to minimize the total cost of all MUs under the latency and resource constraints. Accordingly, we formulate the problem as a Mixed Integer Non-linear Programming (MINLP) problem. In order to solve this problem, the Deep Deterministic Policy Gradient (DDPG)-based algorithm is proposed. The simulation results show that the proposed algorithm is better than other benchmark schemes in different scenarios.

In Vehicular Edge Computing Network (VECN) scenarios, the mobility of vehicles causes the uncertainty of channel state information, which makes it difficult to guarantee the Quality of Service (QoS) in the process of computation offloading and the resource allocation of a Vehicular Edge Computing Server (VECS). A multi-user computation offloading and resource allocation optimization model and a computation offloading and resource allocation algorithm based on the Deep Deterministic Policy Gradient (DDPG) are proposed to address this problem. Firstly, the problem is modeled as a Mixed Integer Nonlinear Programming (MINLP) problem according to the optimization objective of minimizing the total system delay. Then, in response to the large state space and the coexistence of discrete and continuous variables in the action space, a reinforcement learning algorithm based on DDPG is proposed. Finally, the proposed method is used to solve the problem and compared with the other three benchmark schemes. Compared with the baseline algorithms, the proposed scheme can effectively select the task offloading mode and reasonably allocate VECS computing resources, ensure the QoS of task execution, and have a certain stability and scalability. Simulation results show that the total completion time of the proposed scheme can be reduced by 24–29% compared with the existing state-of-the-art techniques.

This paper considers the computation offloading and resource allocation for system sum cost minimization in a mobile edge computing (MEC)network. The computation offloading decision of the user equipments (UEs)can be made through wireless link to access the MEC server. Our intent is to minimize the weighted sum cost of the system, which takes into account the energy consumption and time delay, for both task processing and data transmissions, subject to the force delay requirement of the devices. We first formulate the problem as a mixed-integer and non-convex optimization scheme. Enlightened by the superior performance of reinforcement learning (RL)on solving resource control problems, we put forward a novel RL-based scheme for computation offloading and resource allocation in MEC network. Specifically, the MEC node is considered as an “agent”, which first makes decision on whether to offloading the computing task to the MEC server, and then utilizes convex optimization algorithms to settle the radio spectrum and computation resource scheduling problem in each decision phase. The simulation results present that the RL-based method can achieve better computation offloading and resource allocation performance compared to those of local computing and remote computing modes. It also achieves very similar or slightly better performance compared to the genetic algorithm based method.

Mobile devices may use mobile edge computing to improve energy efficiency and responsiveness by offloading computation tasks to edge servers. However, the transmissions of mobile devices may result in interference that decreases the upload rate and prolongs transmission delay. Clustering has been shown as an effective approach to improve the transmission efficiency for dense devices, but there is no distributed algorithm for the optimization of clustering and computation offloading. In this work, we study the optimization problem of computation offloading to minimize the energy consumption of mobile devices in mobile edge computing by adaptively clustering devices to improve the transmission efficiency. To address the optimization problem in a distributed manner, the decision problem of clustering and computation offloading for mobile devices is formulated as a potential game. We introduce the construction of the potential game and show the existence of Nash equilibrium in the game with a finite enhancement ability. Then, we propose a distributed algorithm of clustering and computation offloading based on game theory. We conducted a simulation to evaluate the proposed algorithm. The numerical results from our simulation show that our algorithm can improve offloading efficiency for mobile devices in mobile edge computing by improving transmission efficiency. By offloading more tasks to edge servers, both the energy efficiency of mobile devices and the responsiveness of computation-intensive applications can be improved simultaneously.

This thesis collects a selective set of outcomes of a PhD course in Electronics, Telecommunications, and Information Technologies Engineering and it is focused on designing techniques to optimize computational resources in different wireless communication environments. Mobile Edge Computing (MEC) is a novel and distributed computational paradigm that has emerged to address the high users demand in 5G. In MEC, edge devices can share their resources to collaborate in terms of storage and computation. One of the computational sharing techniques is computation offloading, which brings a lot of advantages to the network edge, from lower communication, to lower energy consumption for computation. However, the communication among the devices should be managed such that the resources are exploited efficiently. To this aim, in this dissertation, computation offloading in different wireless environments with different number of users, network traffic, resource availability and devices' location are analyzed in order to optimize the resource allocation at the network edge. To better organize the dissertation, the studies are classified in four main sections. In the first section, an introduction on computational sharing technologies is given. Later, the problem of computation offloading is defined, and the challenges are introduced. In the second section, two partial offloading techniques are proposed. While in the first one, centralized and distributed architectures are proposed, in the second work, an Evolutionary Algorithm for task offloading is proposed. In the third section, the offloading problem is seen from a different perspective where the end users can harvest energy from either renewable sources of energy or through Wireless Power Transfer. In the fourth section, the MEC in vehicular environments is studied. In one work a heuristic is introduced in order to perform the computation offloading in Internet of Vehicles and in the other a learning-based approach based on bandit theory is proposed.

Mobile edge computing (MEC) has appeared as a promising solution to fill the gap between the growing computationally intensive applications and limited computation capability of mobile devices by providing powerful computing services at the edge of the wireless access network. To use the services provided by the MEC more effectively, making efficient and reasonable offloading decisions is crucial. In this paper, we study the computation offloading of tasks from multiple users to a single-cell edge server under a dynamic environment. We consider a practical case wherein a group of mobile users with random mobility patterns use a common set of time-varying stochastic transmission channels to perform computation offloading, and the number of active users in the system randomly changes. To reduce the mutual interference among users when accessing the wireless channels, we adopt game theory to formulate the users’ computation offloading decision process as a stochastic game model. Next, we prove the existence of the Nash Equilibrium (NE) for the proposed game model by showing its equivalency to a weighted potential game which has at least one pure-strategy NE point. Then, we present distributed computation offloading algorithms by adopting a payoff-based multi-agent reinforcement learning (MARL) approach to reach the NE of the game. Finally, through simulation, we validate the effectiveness of the proposed algorithms by comparing them with the results obtained from other previously studied multi-agent learning algorithms as well as conventional Q-learning and deep Q-learning algorithms.

Mobile edge computing (MEC) emerges recently as a promising solution to relieve resource-limited mobile devices from computation-intensive tasks, which enables devices to offload workloads to nearby MEC servers and improve the quality of computation experience. In this paper, an MEC enabled multi-user multi-input multi-output (MIMO) system with stochastic wireless channels and task arrivals is considered. In order to minimize long-term average computation cost in terms of power consumption and buffering delay at each user, a deep reinforcement learning (DRL)-based dynamic computation offloading strategy is investigated to build a scalable system with limited feedback. Specifically, a continuous action space-based DRL approach named deep deterministic policy gradient (DDPG) is adopted to learn decentralized computation offloading policies at all users respectively, where local execution and task offloading powers will be adaptively allocated according to each user’s local observation. Numerical results demonstrate that the proposed DDPG-based strategy can help each user learn an efficient dynamic offloading policy and also verify the superiority of its continuous power allocation capability to policies learned by conventional discrete action space-based reinforcement learning approaches like deep Q-network (DQN) as well as some other greedy strategies with reduced computation cost. Besides, power-delay tradeoff for computation offloading is also analyzed for both the DDPG-based and DQN-based strategies.

Collaborative computation offloading and wireless charging scheduling in multi-UAV-assisted MEC networks: A TD3-based approach

Deep Neural Networks meet computation offloading in mobile edge networks: Applications, taxonomy, and open issues

With the emergence of various new Internet of Things (IoT) devices and the rapid increase in the number of users, enormous services and complex applications are growing rapidly. However, these services and applications are resource-intensive and data-hungry, requiring satisfactory quality-of-service (QoS) and network coverage density guarantees in sparsely populated areas, whereas the limited battery life and computing resources of IoT devices will inevitably become insufficient. Unmanned aerial vehicle (UAV)-enabled mobile edge computing (MEC) is one of the most promising solutions that ensures the stability and expansion of the network coverage area for these applications and provides them with computational capabilities. In this paper, computation offloading and resource allocation are jointly considered for multi-user multi-UAV-enabled mobile edge-cloud computing systems. First, we propose an efficient resource allocation and computation offloading model for a multi-user multi-UAV-enabled mobile edge-cloud computing system. Our proposed system is scalable and can support increases in network traffic without performance degradation. In addition, the network deploys multi-level mobile edge computing (MEC) technology to provide the computational capabilities at the edge of the radio access network (RAN). The core network is based on software-defined networking (SDN) technology to manage network traffic. Experimental results demonstrate that the proposed model can dramatically boost the system performance of the system in terms of time and energy.

Dual-timescale resource management for multi-type caching placement and multi-user computation offloading in Internet of Vehicle

Mobile edge computing (MEC), as a new distributed computing model, satisfies the low energy consumption and low latency requirements of computation-intensive services. The task offloading of MEC has become an important research hotspot, as it solves the problems of insufficient computing capability and battery capacity of Internet of things (IoT) devices. This study investigates task offloading scheduling in a dynamic MEC system. By integrating energy harvesting technology into IoT devices, we propose a hybrid energy supply model. We jointly optimize local computing, offloading duration, and edge computing decisions to minimize system cost. On the basis of stochastic optimization theory, we design an online dynamic task offloading algorithm for MEC with a hybrid energy supply called DTOME. DTOME can make task offloading decisions by weighing system cost and queue stability. We quote dynamic programming theory to obtain the optimal task offloading strategy. Simulation results verify the effectiveness of DTOME, and show that DTOME entails lower system cost than two baseline task offloading strategies.

UAV-mounted IRS assisted wireless powered mobile edge computing systems: Joint beamforming design, resource allocation and position optimization