Joint Microservices Caching and Task Offloading Framework in VEC Based on Deep Reinforcement Learning

Vehicular edge computing (VEC) has greatly enhanced the quality of vehicle service with low latency and high reliability. However, in some areas not covered by roadside infrastructures or in cases when the infrastructures are damaged or fail, the offloaded tasks cannot have the chance to be performed. Even in the areas deployed with infrastructures, when a large number of offloaded tasks are generated, the edge servers may not be capable of processing them in time, owing to their computing resources constraint. Based on the above observations, we proposed the idea of parked vehicle cooperation in VEC, which uses roadside parked vehicles with underutilized computational resources to cooperate with each other to perform the compute-intensive tasks. Our approach aims to overcome the challenge brought by infrastructure lacking or failure and make up for the shortage of computing resources in VEC. In our approach, firstly, the roadside parked vehicles are managed as different parking clusters. Then, the optimal amount of resources required for each offloaded task is analyzed. Furthermore, a task offloading algorithm based on deep reinforcement learning (DRL) is proposed to minimize the total cost, which is composed of the task execution delay and the energy consumption overhead of the parked vehicles for executing the task. A large number of simulation results show that, compared with other algorithms, our approach not only has the highest task completion execution successful rate, but also has the lowest task execution cost.

Taking the mobile edge computing paradigm as an effective supplement to the vehicular networks can enable vehicles to obtain network resources and computing capability nearby, and meet the current large-scale increase in vehicular service requirements. However, the congestion of wireless networks and insufficient computing resources of edge servers caused by the strong mobility of vehicles and the offloading of a large number of tasks make it difficult to provide users with good quality of service. In existing work, the influence of network access point selection on task execution latency was often not considered. In this paper, a pre-allocation algorithm for vehicle tasks is proposed to solve the problem of service interruption caused by vehicle movement and the limited edge coverage. Then, a system model is utilized to comprehensively consider the vehicle movement characteristics, access point resource utilization, and edge server workloads, so as to characterize the overall latency of vehicle task offloading execution. Furthermore, an adaptive task offloading strategy for automatic and efficient network selection, task offloading decisions in vehicular edge computing is implemented. Experimental results show that the proposed method significantly improves the overall task execution performance and reduces the time overhead of task offloading.

Joint task offloading and data caching in mobile edge computing networks

Edge Computing is a distributed computing architecture where the computing process takes place near the user’s physical location or at the source where the data originates. By placing the edge servers (ES) closer to the user’s location, the services received are faster and more reliable while benefiting from edge computing. Various Internet of Vehicle (IoV) applications in smart cities including assisted and autonomous driving, real-time accidents monitoring require huge data processing and low-latency communication. Since the user’s devices are resource constrained, an effective approach to address this constraint is to offload their tasks to nearby ES. So, an adaptive placements (to respond continuously changing environment) of these edge computing devices play a major role in the performance of various IoV applications. So, an efficient placement of ES is considered a critical issue in vehicular edge computing (VEC). To address the efficient and cost aware dynamic ES placement problem (CADEP), we developed two greedy algorithms. First, is cost aware and vehicle density based deployment of ES (static) that ensures that each vehicle’s demand is covered by at least by one ES (coverage constraint), called Greedy_static. Second, is based on vehicle density and is dynamic as per changing environment, called Greedy_dynamic which updates ES locations periodically based on change in the environment. To minimize the relocation cost, we formulated an optimization problem and used Hungarian matching to find optimal cost. For various vehicle densities, we found that our algorithms outperform uniform strategies in terms of cost-effectiveness and ES utilization. Further, for dynamic relocation of ES, we have shown that the cost required to relocate ES randomly is more as compared to our proposed algorithm Greedy_dynamic.

Vehicular Edge Computing (VEC) brings the computational resources in close proximity to the service requestors and thus supports explosive computing demands from smart vehicles. However, the limited computing capability of VEC cannot simultaneously respond to large amounts of offloading requests, thus restricting the performance of VEC system. Besides, a mass of traffic data can incur tremendous pressure on the front-haul links between vehicles and the edge server. To strengthen the performance of VEC, in this paper we propose to place services beforehand at the edge server, e.g., by deploying the services/tasks-oriented data (e.g., related libraries and databases) in advance at the network edge, instead of downloading them from the remote data center or offloading them from vehicles during the runtime. In this paper, we formulate the service placement problem in VEC to minimize the average response latency for all requested services along the slotted timeline. Specifically, the time slot spanned optimization problem is converted into per-slot optimization problems based on the Lyapunov optimization. Then a greedy heuristic is introduced to the drift-plus-penalty-based algorithm for seeking the approximate solution. The simulation results reveal its advantages over others in terms of optimal values and our strategy can satisfy the long-term energy constraint.

Complex vehicular applications, such as automatic driving and augmented reality are delay sensitive and require massive computational resources. Despite being more connected and smarter, vehicles still cannot appropriately meet the demands of these applications. By allowing neighboring vehicles and edge servers coupled to base stations to share their available computing resources, vehicular edge computing systems help to handle these applications. Then, vehicles can use the task offloading technique by sending application tasks to be executed remotely and receiving the processing results later. Although this technique aims to reduce application execution time, performing it in vehicular scenarios is challenging. In such scenarios, network nodes vary their computing and energy loads and move quickly, causing frequent disconnections and failures. Thus, we propose an algorithm called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Bee colony-based Task offloading in Vehicular edge computing</i> (BTV) to reliably reduce the execution time of applications in vehicular edge computing systems. The BTV algorithm provides task scheduling solutions to different servers in a feasible time, using several contextual parameters and wireless access in vehicular environments and fifth-generation networks. Experimental results show that our solution can reduce the average execution time of applications by up to 74.4% and with up to 0.0% of failures, outperforming other existing solutions.

ABSTRACTThe increasing processing demands of vehicle applications pose a major challenge for the Internet of Vehicles (IoVs). Multi‐access edge computing (MEC) offloads computation‐intensive activities to edge servers, significantly enhancing computing capacity and extending vehicle battery life. Nevertheless, traditional offloading strategies fail to effectively balance computation tasks among vehicles and edge servers, resulting in suboptimal resource utilization and increased energy consumption. Therefore, an Energy‐Efficient Computation Offloading Scheme for Vehicular Edge Computing utilizing Improved Golden Eagle Optimization Algorithm (EECO‐VEC‐IGEOA) is proposed. An Improved Golden Eagle Optimization Algorithm (IGEOA) is designed to optimize energy‐efficient computation offloading in vehicular edge computing networks (VECN). By dynamically allocating computation tasks between vehicles and edge servers depending on real‐time conditions, the EECO‐VEC‐IGEOA model aims to improve overall network performance and energy efficiency. The EECO‐VEC‐IGEOA method reduces energy consumption by 14.62%, 16.84%, and 19.16%, and task completion time by 15.84%, 18.92%, and 20.69% compared to the existing approaches.

In vehicular edge computing (VEC) systems, vehicles can contribute their computing resources to the network, and help other vehicles or pedestrians to process their computation tasks. However, the high mobility of vehicles leads to a dynamic and uncertain vehicular environment, where the network topologies, channel states and computing workloads vary fast across time. Therefore, it is challenging to design task offloading algorithms to optimize the delay performance of tasks. In this chapter, we consider the task offloading among vehicles, and design learning-based task offloading algorithms based on the multi-armed bandit (MAB) theory, which enable vehicles to learn the delay performance of their surrounding vehicles while offloading tasks. We start from the single offloading case where each task is offloaded to one vehicle to be processed, and propose an adaptive learning-based task offloading (ALTO) algorithm, by jointly considering the variations of surrounding vehicles and the input data size. To further improve the reliability of the computing services, we introduce the task replication technique, where the replicas of each task is offloaded to multiple vehicles and processed by them simultaneously, and propose a learning-based task replication algorithm (LTRA) based on combinatorial MAB. We prove that the proposed ALTO and LTRA algorithms have bounded learning regret, compared with the genie-aided optimal solution. And we also build a system level simulation platform to evaluate the proposed algorithms in the realistic vehicular environment.

Vehicular Edge Computing (VEC), based on the Edge Computing motivation and fundamentals, is a promising technology supporting Intelligent Transport Systems services, smart city applications, and urban computing. VEC can provide and manage computational resources closer to vehicles and end-users, providing access to services at lower latency and meeting the minimum execution requirements for each service type. This survey describes VEC’s concepts and technologies; we also present an overview of existing VEC architectures, discussing them and exemplifying them through layered designs. Besides, we describe the underlying vehicular communication in supporting resource allocation mechanisms. With the intent to overview the risks, breaches, and measures in VEC, we review related security approaches and methods. Finally, we conclude this survey work with an overview and study of VEC’s main challenges. Unlike other surveys in which they are focused on content caching and data offloading, this work proposes a taxonomy based on the architectures in which VEC serves as the central element. VEC supports such architectures in capturing and disseminating data and resources to offer services aimed at a smart city through their aggregation and the allocation in a secure manner.

The vehicular edge computing (VEC) system integrates the computing resources of vehicles, and provides computing services for other vehicles and pedestrians with task offloading. However, the vehicular task offloading environment is dynamic and uncertain, with fast varying network topologies, wireless channel states and computing workloads. These uncertainties bring extra challenges to task offloading. In this work, we consider the task offloading among vehicles, and propose a solution that enables vehicles to learn the offloading delay performance of their neighboring vehicles while offloading computation tasks. We design an adaptive learning-based task offloading (ALTO) algorithm based on the multi-armed bandit (MAB) theory, in order to minimize the average offloading delay. ALTO works in a distributed manner without requiring frequent state exchange, and is augmented with input-awareness and occurrence-awareness to adapt to the dynamic environment. The proposed algorithm is proved to have a sublinear learning regret. Extensive simulations are carried out under both synthetic scenario and realistic highway scenario, and results illustrate that the proposed algorithm achieves low delay performance, and decreases the average delay up to 30% compared with the existing upper confidence bound based learning algorithm.

Cloud based vehicular edge computing is a promising technology to deliver quality of services for accessing public cloud from vehicles. As in-vehicle computers (e.g. onboard Android devices) are typically energy and resource constrained, they need to operate energy-efficiently. Due to the high cost in building sustainable infrastructure to support vehicular edge computing systems, edge services must also operate with high energy-efficiency for better acceptability and applicability in the market place. In this paper, we present a novel situation-enabled framework to enhance energy-efficiency of both in-vehicle computer applications and Mobile Edge Computing (MEC) services. The framework consists of three components. First, we collect a user's in-vehicle driving situation data and use the data to train Long Short Term Memory (LSTM) deep learning model. The model is used to predict the user's future situation, determine whether the requested application is allowable, and manage the allowable application pertaining to the predicted situation. Second, from vehicles to MEC servers, we optimize the energy consumption in request routing. Third, a Breadth First Search (BFS) based offloading algorithm to coordinate the placement of servers in the MEC servers to save the energy consumption in the server pool. Implementation of LSTM on top of a situation model and simulation of the vehicular edge offloading have been conducted to validate the energy efficiency of the proposed framework. The results show that the performance of our proposed offloading algorithm can be at par with the most aggressive energy saving variant.

Vehicular Edge Computing (VEC) is a promising paradigm to enable huge amount of multimedia content to be cached in proximity to vehicles. Since vehicles are equipped with a certain amount of caching resource, they can be regarded as edge nodes to expand the caching capacity of the network edge. However, with much sensitive personal information, vehicles may be not willing to cache their content to an untrusted vehicle. Permission blockchain has the potential to address such an issue. In this paper, we utilize permissioned blockchain to design a secure content caching scheme between vehicles. Since high mobility of vehicles makes a dynamic caching environment, we exploit deep reinforcement learning approach to design the content caching scheme. Moreover, we propose a new block verifier selection metric, Proof-of-Utility (PoU), to enable a lightweight permissioned blockchain. Security analysis shows that our proposed blockchain empowered content caching can achieve security and privacy protection. Numerical results based on the Uber dataset indicate the DRL-inspired content caching scheme significantly outperforms two benchmark policies.

Vehicular edge computing (VEC) emerges as a promising paradigm for processing computing-intensive parallel vehicular tasks, where vehicular tasks can be offloaded to the edge nodes [e.g., roadside units (RSUs)] to seek less computing delay. Considering the impact of computation services on offloading efficiency, there are several works that jointly study the decision making of task offloading and service caching. However, the existing works fail to consider the time-varying service requests and ignore the time-slots correlation of the computation services. To bridge the gap, this work designs a service-aware parallel task offloading approach, which is the first work to jointly explore time-varying computation services and task offloading based on real-world vehicular trajectory data in VEC networks. Specifically, we first propose a computation service prediction algorithm using the real-world vehicular trajectory data. Guided by this, RSUs flexibly precache computation services. Then, we propose a learning-based parallel task offloading algorithm, which allows vehicles to make offloading decisions based on the history of the edge selections. Furthermore, we conduct simulations to validate the proposed algorithm. The results demonstrate that the proposed algorithm reduces task delay by 45%, 58%, and 55% compared to the algorithms without service-aware computation offloading under various CPU cycles, task numbers, and time slots.

Vehicle Edge Computing (VEC) leverages compact cloud computing at the mobile network edge to meet the processing and latency needs of vehicles. By bringing computation closer to the vehicles, VEC reduces data transmission, minimizes latency, and boosts performance for compute-intensive applications. However, during peak hours of urban road traffic, the scarce computational resources available at edge servers could pose challenges in fulfilling the processing needs of vehicles. Introducing Unmanned Aerial Vehicles (UAVs) as supplementary edge computing nodes could significantly mitigate the aforementioned issue. In this paper, we propose a flexible edge computing framework in which a fleet of UAVs function as mobile computational service providers, offering computation offloading services to multiple vehicles. We design and optimize a computation offloading model for the UAV-enabled vehicle edge computing environment. The proposed model tackles the task offloading challenge, aiming to optimize UAV revenue and task processing efficiency while considering the constraints of UAVs’ restricted computational power and energy resources. Towards this end, our model jointly considers two key factors: task partitioning and computational resource allocation. To tackle the challenges posed by the aforementioned non-convex optimization problem, we construct a Markov Decision Process (MDP) model for the multi-UAV-enabled mobile edge computing system and introduce an innovative Multi-Agent Deep Reinforcement Learning (MADRL) framework addressing the decision-making challenge represented by MDP model. Comprehensive simulation outcomes illustrate that our devised task offloading technique outperforms other optimization methods.

While service-oriented low Earth orbit (LEO) satellite edge computing (LSEC) frameworks enable diverse edge services for user tasks, the heterogeneous distribution of terrestrial users causes substantial imbalance in computational task loads across satellites. This asymmetric workload leads to inefficient resource utilization and degraded edge computing performance. To address these challenges, we propose a joint optimization framework that integrates edge service deployment and task offloading, supported by service popularity analysis and spatiotemporal user-task modeling. The framework employs a two-timescale design: at the large timescale, an improved atomic orbital search (iAOS) heuristic dynamically optimizes service placement, configuration, and resource allocation; at the small timescale, a direction-selective multi-agent double deep Q-network (DS-MDDQN) leverages deep reinforcement learning to route tasks to the most suitable processing nodes. Extensive simulations show that our approach significantly outperforms six representative baselines in both user-perceived performance and system-level efficiency. Replacement studies further verify the effectiveness of each component: iAOS enhances resource utilization and reduces task failure through optimized service deployment, while DS-MDDQN mitigates network dynamics and lowers task completion latency via adaptive task offloading.