Multi-agent deep reinforcement learning for task offloading in group distributed manufacturing systems

The rapid evolution of 6G networks has brought new challenges in the domain of task offloading (TO), particularly within edge computing environments that are heavily reliant on the internet of things (IoT). Traditional TO methods that based on rule-based heuristics or shallow learning techniques fail to adapt efficiently to the dynamic, unpredictable network conditions, resource heterogeneity, and varying task demands. The proliferation of edge computing, the IoT, and 6G networks has introduced new challenges in TO due to dynamic network conditions, resource heterogeneity, and unpredictable task demands. To address these challenges, this work proposes an innovative TO method that integrates deep reinforcement learning (DRL) with heuristic search methods. The combination of DRL and heuristic algorithms enhances adaptability, convergence speed, and decision-making efficiency, making it well-suited for real-time TO in complex and unpredictable environments This paper proposes a novel hybrid TO framework that integrates DRL with genetic algorithms (GA) to address these challenges. The proposed hybrid optimization technique offer promising solutions by leveraging the strengths of individual approaches to balance competing objectives, such as energy consumption, task completion time, and resource utilization. This method explores optimization strategies to enhance TO efficiency in decentralized environments mainly focusing on optimizing energy use while ensuring performance metrics like latency, throughput, and task deadlines are met.

With the ongoing advancement of electric power Internet of Things (IoT), traditional power inspection methods face challenges such as low efficiency and high risk. Unmanned aerial vehicles (UAVs) have emerged as a more efficient solution for inspecting power facilities due to their high maneuverability, excellent line-of-sight communication capabilities, and strong adaptability. However, UAVs typically grapple with limited computational power and energy resources, which constrain their effectiveness in handling computationally intensive and latency-sensitive inspection tasks. In response to this issue, we propose a UAV task offloading strategy based on deep reinforcement learning (DRL), which is designed for power inspection scenarios consisting of mobile edge computing (MEC) servers and multiple UAVs. Firstly, we propose an innovative UAV-Edge server collaborative computing architecture to fully exploit the mobility of UAVs and the high-performance computing capabilities of MEC servers. Secondly, we established a computational model concerning energy consumption and task processing latency in the UAV power inspection system, enhancing our understanding of the trade-offs involved in UAV offloading strategies. Finally, we formalize the task offloading problem as a multi-objective optimization issue and simultaneously model it as a Markov Decision Process (MDP). Subsequently, we proposed a task offloading algorithm based on a Deep Deterministic Policy Gradient (OTDDPG) to obtain the optimal task offloading strategy for UAVs. The simulation results demonstrated that this approach outperforms baseline methods with significant improvements in task processing latency and energy consumption.

Unmanned Aerial Vehicles (UAVs) are used in various applications, including crowd management, crime prevention, accident detection, and rescue operations. However, since UAVs perform their tasks independently, some UAV applications are dynamic and geographically distributed, which may require extensive real-time processing capabilities. Thus, processing UAV data locally can be challenging due to their limited computing capabilities. To overcome such limitations, fog and cloud computing can facilitate UAV application development by providing additional resource capacities when needed. Despite this, designing sophisticated and efficient UAV task offloading strategies that collaborate with fog and cloud technologies considering their service latency and energy consumption, is rarely addressed in the literature. Therefore, a collaborative offloading strategy for UAV applications is presented in this work, leveraging fog and cloud computing advantages and capabilities. This approach aims to minimize UAVs’ service latency and energy consumption, as well as provide the required resources and services in real time. In addition, task offloading decisions are formulated using the Mixed-Integer Linear Programming (MILP) model to reduce the energy consumption of the entire UAV-fog-cloud system by optimizing the allocation of computation resources and communication requested by each UAV. The simulation results demonstrate that the proposed strategy can significantly reduce UAV service latency by 15.38%, 35.29%, and 59.26%, as well as decrease overall energy consumption (including processing and networking) by 3.3%, 7.37%, and 12% when compared to alternative standalone strategies (namely UAV, fog, and cloud).

Large number of Internet of Things (IoT) devices deployed on demand to monitor surroundings and offload computation-intensive tasks to edge servers, which favors low-latency IoT applications. However, some of them scatter in areas with limited or no available communication infrastructure (no-ACI), which rises a huge challenge for task offloading, due to when and which devices have tasks cannot be known in advance. In this article, a trust-based active notice task offloading (TANTO) scheme in the aerial computing system (ACS) assisted by the unmanned aerial vehicle (UAV) is proposed to provide trust and low-delay task offloading for resource-limited IoT devices in areas with no-ACI. The main innovations of TANTO in ACS are as follow: 1) a novel task offloading mechanism for IoT devices in networks with no-ACI is proposed, where devices broadcast task offloading notice, making it easy for UAV to know where tasks need to offload without aimless search, which can effectively reduce UAV’s flight distance and task completion delay at the same time; 2) a trust calculation and reasoning method is proposed to calculate network trust, and divide the network into grids with different trust values, which guides UAV through grids with higher trust to increase the task completion rate and reduce delay; and 3) an online UAV trajectory optimization algorithm is proposed to dynamically prioritize tasks with urgent deadline. A large number of experimental results show that TANTO has better performance than previous studies in terms of task completion rate, tasks’ average completion time, and UAV’s flight cost.

Mobile edge computing (MEC) can effectively overcome the shortcomings of high-latency in mobile cloud computing (MCC) by deploying the cloud resources, e.g., storage and computational capability, to the edge. However, the limited computation capability of the MEC restricts the scalability of offloading. Therefore, the basic requirements of the MEC system are to explore effective offloading decisions and resource allocation methods. To address it, we develop a collaborative computing system composed of local computing (mobile device), MEC (edge cloud) and MCC (central cloud). Based on the proposed collaborative computing system, we design a novel Q-learning based computation offloading (QLCOF) policy to achieve the optimal resource allocation and offloading scheme by prescheduling the computation side for each task from a global perspective. Specifically, we first model the offloading decision process as a Markov decision process (MDP) and design a state loss function (STLF) to measure the quality of experience (QoE). After that, we define the cumulation of STLFs as the system loss function (SYLF) and formulate an SYLF minimization problem. Due to the difficulty to directly solve the formulated problem, we decompose it into multiple subproblems and preferentially optimize the transmission power and computation frequency of the edge cloud by the quasi-convex bisection and polynomial analysis method, respectively. Based on the precalculated offline transmission power and edge cloud computation frequency, we develop a Q-learning based offloading (QLOF) scheme to minimize the SYLF by optimizing offloading decisions. Finally, the numeral results show that the proposed QLOF scheme effectively reduces the SYLF under different parameters.

Accelerating the development of the 5G network and Internet of Things (IoT) application, multi-access edge computing (MEC) in a small-cell network (SCN) is designed to provide computation-intensive and latency-sensitive applications through task offloading. However, without collaboration, the resources of a single MEC server are wasted or sometimes overloaded for different service requests and applications; therefore, it increases the user’s task failure rate and task duration. Meanwhile, the distinct MEC server has faced some challenges to determine where the offloaded task will be processed because the system can hardly predict the demand of end-users in advance. As a result, the quality-of-service (QoS) will be deteriorated because of service interruptions, long execution, and waiting time. To improve the QoS, we propose a novel Fuzzy logic-based collaborative task offloading (FCTO) scheme in MEC-enabled densely deployed small-cell networks. In FCTO, the delay sensitivity of the QoS is considered as the Fuzzy input parameter to make a decision where to offload the task is beneficial. The key is to share computation resources with each other and among MEC servers by using fuzzy-logic approach to select a target MEC server for task offloading. As a result, it can accommodate more computation workload in the MEC system and reduce reliance on the remote cloud. The simulation result of the proposed scheme show that our proposed system provides the best performances in all scenarios with different criteria compared with other baseline algorithms in terms of the average task failure rate, task completion time, and server utilization.

With the rapid development and convergence of the mobile Internet and the Internet of Things (IoT), the diverse IoT applications with requirements of ultra-low latency and heterogeneous connectivity have been increasing sharply in recent years. In this work, we consider the problem of diverse task offloading and computation resource allocation in the dynamic and heterogeneous MEC-enable network, wherein heterogeneous characterized by the MEC servers with different computing capacities and multi-type applications with different computation requirements. Firstly, to drive the long-term system-wide effectiveness towards a near-optimum, joint task offloading and computation resource allocation is proposed by invoking the Lyapunov optimization theory, and we analyze the tradeoff between task offloading revenue and latency. Besides, a heuristic task offloading methodology based on a search tree is developed to get the optimal offloading strategy in a heterogeneous environment. Moreover, to improve processing efficiency and reduce unnecessary communication overhead, the Offloading Priority Selection Criterion (OPSC) is designed. Finally, the effectiveness and rationality of the algorithm are verified by experimental simulations.

Task offloading in mobile edge computing (MEC) becomes particularly challenging when dealing with complex, interdependent subtasks, especially under dynamic network conditions and resource constraints. This challenge is central to enabling intelligent and adaptive behavior in holographic counterparts, real-time virtual replicas of physical consumer Internet of Things (IoT) devices such as smart wearables, home automation units, or augmented reality systems. In this paper, we propose HiDeR-GQ, a novel Hierarchical Dependency Reduction framework that integrates quantized graph-based deep reinforcement learning (DRL) which acts as the decision-making core of holographic counterparts deployed in heterogeneous edge networks. The core of HiDeR-GQ is a customized deep reinforcement learning agent that integrates a quantized Graph Attention Network (GATv2) for structure-aware state encoding of task dependency graphs, and a personalized federated Double Deep Q-Network (DDQN) to enable collaborative learning across distributed edge nodes while accounting for heterogeneity. To further optimize performance, we introduce a latency-aware dependency pruning mechanism that hierarchically simplifies task graphs by removing non-critical dependencies to reduce end-to-end latency without compromising task accuracy. We evaluate HiDeR-GQ in a simulated MEC environment involving multiple edge servers and users executing directed acyclic graph (DAG) structured application tasks. The results demonstrate that HiDeR-GQ achieves up to 25% lower average task completion delay, higher cumulative reward, and reduced communication cost compared to state-of-the-art offloading strategies. These benefits come with minimal trade-offs in decision optimality, highlighting HiDeR-GQ’s efficiency and scalability for next-generation IoT and edge intelligence applications.

Vehicle Edge Computing (VEC) leverages compact cloud computing at the mobile network edge to meet the processing and latency needs of vehicles. By bringing computation closer to the vehicles, VEC reduces data transmission, minimizes latency, and boosts performance for compute-intensive applications. However, during peak hours of urban road traffic, the scarce computational resources available at edge servers could pose challenges in fulfilling the processing needs of vehicles. Introducing Unmanned Aerial Vehicles (UAVs) as supplementary edge computing nodes could significantly mitigate the aforementioned issue. In this paper, we propose a flexible edge computing framework in which a fleet of UAVs function as mobile computational service providers, offering computation offloading services to multiple vehicles. We design and optimize a computation offloading model for the UAV-enabled vehicle edge computing environment. The proposed model tackles the task offloading challenge, aiming to optimize UAV revenue and task processing efficiency while considering the constraints of UAVs’ restricted computational power and energy resources. Towards this end, our model jointly considers two key factors: task partitioning and computational resource allocation. To tackle the challenges posed by the aforementioned non-convex optimization problem, we construct a Markov Decision Process (MDP) model for the multi-UAV-enabled mobile edge computing system and introduce an innovative Multi-Agent Deep Reinforcement Learning (MADRL) framework addressing the decision-making challenge represented by MDP model. Comprehensive simulation outcomes illustrate that our devised task offloading technique outperforms other optimization methods.

As a new network technology, mobile-edge computing (MEC) combined with the Internet of Vehicles (IoV) can effectively improve the efficiency of task computing and offloading. However, the power of edge computing will be severely limited to the areas with poor MEC server coverage. Furthermore, there are a number of peripheral vehicles with temporarily idle computing resources on the road, so how to put the resources of these vehicles into use becomes the primary issue to be considered. In this article, a distributed multihop task offloading decision model for task execution efficiency is developed, which mainly consists of two parts: 1) a candidate vehicle selection mechanism for screening the neighboring vehicles that can participate in offloading and 2) a task offloading decision algorithm for obtaining the task offloading solution. Considering the impact of different hop and wireless communication ranges on communication ranges on task completion in a generic scenario, we introduce the hop count <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> and select the neighboring vehicles in the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -hop wireless communication range as the candidate vehicles. Then, the problem of offloading is modeled as a generalized allocation model with constraints which is solved by the greedy algorithm and discrete bat algorithm, respectively. The results show that compared with the scheme in which the task vehicle randomly selects the neighboring vehicles to offload and the scheme that all tasks are completed locally, the offloading scheme in which all tasks are completed under the greedy algorithm or bat-based algorithm has advantages in time delay performance in terms of different task number, task required computation power, and task size environment. Besides, this article also explores the influence of hop count <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> on the results when selecting candidate vehicles from the neighboring vehicles within the range of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> hop. The results show that the increase of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> will also increase the number of candidate vehicles, which makes the time delay lower. Under the parameters set in this article, the time delay required for the greedy algorithm offloading scheme to complete all tasks is a lower bound on the time delay of the bat algorithm scheme. The greedy algorithm scheme reduces latency by 0.2–2.4 s compared to the scheme where tasks are all completed locally, and it reduces latency by 0.16–2.3 s compared to the random offloading scheme.

Mobile edge computing (MEC), as a new distributed computing model, satisfies the low energy consumption and low latency requirements of computation-intensive services. The task offloading of MEC has become an important research hotspot, as it solves the problems of insufficient computing capability and battery capacity of Internet of things (IoT) devices. This study investigates task offloading scheduling in a dynamic MEC system. By integrating energy harvesting technology into IoT devices, we propose a hybrid energy supply model. We jointly optimize local computing, offloading duration, and edge computing decisions to minimize system cost. On the basis of stochastic optimization theory, we design an online dynamic task offloading algorithm for MEC with a hybrid energy supply called DTOME. DTOME can make task offloading decisions by weighing system cost and queue stability. We quote dynamic programming theory to obtain the optimal task offloading strategy. Simulation results verify the effectiveness of DTOME, and show that DTOME entails lower system cost than two baseline task offloading strategies.

The modeling and analysis of large-scale stochastic MEC systems are of great significance in providing useful design guidelines for practical MEC networks. In most prior works, the users generally adopt the same strategy to select appropriate MEC access points (MAPs), where the fact that the available mobile computing services are different among the randomly distributed users is ignored. To this end, this paper proposes a user-centric cooperative offloading scheme to enable more flexible and efficient user task offloading. Specifically, each user can be served by one or two MAPs based on both the communication performance and computing performance. To evaluate the performance gains acquired from the proposed task offloading scheme, we first derive the service mode assignment probability, link distance distribution, and interference intensity to capture the network characteristics. Further, the moment and the meta distribution of the task transmission performance are analyzed. Based on the above results, we focus on the distribution of the computation workload to evaluate the edge computing service capability. From the analytical and simulation results, it is demonstrated that compared with the non-cooperative offloading scheme, the proposed task offloading scheme not only achieves more reliable and fair task offloading but also increases the computing service capacity.

With the rapid advancement of autonomous driving technology, the increasing computational demands of intelligent vehicles have driven the adoption of cloud and edge computing to augment limited onboard resources. However, this cloud–edge integration presents new challenges for efficient task offloading. In addition, the high mobility of vehicles further complicates the design of reliable offloading strategies. To address these challenges, this paper proposes a Cloud-Edge-Vehicle (CEV) framework that leverages predictable vehicle trajectories for optimized task offloading. A spatio-temporal multi-head self-attention long short-term memory (ST-MHSA LSTM) model is designed to accurately predict vehicle trajectories by capturing motion trends and interactions with neighboring vehicles. Building upon the trajectory prediction information, a deep reinforcement learning (DRL)-based task offloading algorithm is proposed. This algorithm incorporates a dynamic priority assignment strategy to prioritize delay-sensitive tasks according to their urgency, thereby improving offloading performance and reducing system costs associated with task transmission and execution. To mitigate the adverse effects of inevitable trajectory prediction errors, a prediction consistency-based deviation correction strategy is further introduced, enhancing decision robustness in dynamic scenarios. Simulation results show that as task numbers increase, the proposed framework outperforms traditional methods (i.e., local, edge, cloud, and random computing) in task success ratio, processing delay, and energy consumption. Task success ratio improves by 115.79%, 114.44%, 27.09%, and 135.19% over local, edge, cloud, and random computing, respectively. Average processing delay is reduced by 70.47%, 62.23%, 38.69%, and 79.85%, while average energy consumption decreases by 65.33%, 66.59%, 59.12%, and 74.66%. These results highlight the framework’s superior performance for computation-intensive and delay-sensitive vehicular applications.

The rapid growth of connected medical devices generates massive volumes of heterogeneous health data that must be processed and transmitted in real time. In such environments, minimizing latency and energy consumption remains a critical challenge for next-generation health monitoring systems. Existing reinforcement learning and optimization methods for intelligent communication networks face several challenges, including slow convergence, high computational overhead, and inefficiency in handling task prioritization. To resolve these issues, this work develops a chaotic dung beetle optimization-boosted multi-agent deep reinforcement learning that jointly optimizes communication reliability, computational efficiency, and task prioritization. A reward function is designed to jointly minimize delay, energy usage, and system cost while preserving information freshness. Specifically, the dung beetle optimization process is combined with a piecewise linear chaotic map to enhance population diversity, which significantly improves search space exploration and leads to faster convergence and higher solution quality. The proposed algorithm enhances the exploration capability of multi-agent deep reinforcement learning through the integration of chaotic dung beetle optimization, enabling more accurate and reliable decision-making in real-world applications. Extensive experiments demonstrate that the proposed chaotic dung beetle optimization-boosted multi-agent deep reinforcement learning model achieves superior performance compared to baseline algorithms. Specifically, it reaches an accuracy of over 97.00% with rapid convergence, reduces system cost under varying health data sizes and Medical Internet of Things devices, and maintains robust scalability across diverse workloads. Moreover, the model achieves significant reductions in communication latency and energy consumption as central processing unit cycles and bandwidth increase, while effectively prioritizing high-criticality tasks.

The edge-cloud computing paradigm has become a key component of fifth-generation (5G) networks and beyond. It becomes difficult to meet the computation-intensive and delay-sensitive service requirements of intelligent IoT applications by using only edge computing or cloud computing alone. Therefore, an intelligent resource management technique is required to efficiently distribute the task offloading to the edge and cloud computing systems. In this paper, we aim to optimize the task offloading under delay constraints to maximize resource utilization and minimize the offloading rejection in the edge-cloud computing system. Considering optimal decisions for task offloading and resource allocation deep reinforcement learning is applied. We formulate this optimization problem by the Markov decision process and then use Deep Q-Network (DQN) to find the optimal policy for task offloading. Thus, we design the DQN-edge-cloud (DQNEC) computing scheme to update the policy and optimally offload the task in a dynamic edge-cloud environment with considering resource utilization. The simulation results show that DQNEC achieves good performance than the heuristic approaches in terms of maximizing resource utilization with minimum cost and maximizing task offloading with minimum task rejection.