A Multi-Timescale Load Balancing Approach in Vehicular Edge Computing

Vehicle Edge Computing (VEC) leverages compact cloud computing at the mobile network edge to meet the processing and latency needs of vehicles. By bringing computation closer to the vehicles, VEC reduces data transmission, minimizes latency, and boosts performance for compute-intensive applications. However, during peak hours of urban road traffic, the scarce computational resources available at edge servers could pose challenges in fulfilling the processing needs of vehicles. Introducing Unmanned Aerial Vehicles (UAVs) as supplementary edge computing nodes could significantly mitigate the aforementioned issue. In this paper, we propose a flexible edge computing framework in which a fleet of UAVs function as mobile computational service providers, offering computation offloading services to multiple vehicles. We design and optimize a computation offloading model for the UAV-enabled vehicle edge computing environment. The proposed model tackles the task offloading challenge, aiming to optimize UAV revenue and task processing efficiency while considering the constraints of UAVs’ restricted computational power and energy resources. Towards this end, our model jointly considers two key factors: task partitioning and computational resource allocation. To tackle the challenges posed by the aforementioned non-convex optimization problem, we construct a Markov Decision Process (MDP) model for the multi-UAV-enabled mobile edge computing system and introduce an innovative Multi-Agent Deep Reinforcement Learning (MADRL) framework addressing the decision-making challenge represented by MDP model. Comprehensive simulation outcomes illustrate that our devised task offloading technique outperforms other optimization methods.

Facing the requirements of intelligent vehicles for massive data processing, vehicular edge computing (VEC) utilizes computing resources deployed on the roadside infrastructure to provide proximity computing services for vehicles and forms a novel computing paradigm. Thus, vehicles can reduce the burden of local computing and improve computing efficiency by offloading tasks to roadside computing servers or neighboring resource-idle vehicles for execution via cooperative computation offloading (CO). However, dynamic communication channel states and data handover among multiple VEC servers caused by vehicle mobility pose challenges for CO decision-making and data security. This paper applies blockchain to the cooperative CO of VEC and thus proposes a cooperative CO and secure handover framework with a consensus mechanism to guarantee the efficiency of cooperative CO and secure handover. In this framework, models for vehicle mobility and cooperative CO handover are constructed, and a consensus mechanism is proposed. This mechanism ensures the synchronization and immutability of offloaded data in the CO handover. A cooperative CO decision optimization is also formulated considering secure handover with blockchain technology to optimize the latency of vehicular computing tasks. To solve this complex problem, this optimization is transformed into a Markov decision process and a cooperative CO decision algorithm with multiagent deep reinforcement learning is designed, thus achieving the optimal solution. Extensive simulations verified the performance and effectiveness of the proposed method.

The rapid growth of connected medical devices generates massive volumes of heterogeneous health data that must be processed and transmitted in real time. In such environments, minimizing latency and energy consumption remains a critical challenge for next-generation health monitoring systems. Existing reinforcement learning and optimization methods for intelligent communication networks face several challenges, including slow convergence, high computational overhead, and inefficiency in handling task prioritization. To resolve these issues, this work develops a chaotic dung beetle optimization-boosted multi-agent deep reinforcement learning that jointly optimizes communication reliability, computational efficiency, and task prioritization. A reward function is designed to jointly minimize delay, energy usage, and system cost while preserving information freshness. Specifically, the dung beetle optimization process is combined with a piecewise linear chaotic map to enhance population diversity, which significantly improves search space exploration and leads to faster convergence and higher solution quality. The proposed algorithm enhances the exploration capability of multi-agent deep reinforcement learning through the integration of chaotic dung beetle optimization, enabling more accurate and reliable decision-making in real-world applications. Extensive experiments demonstrate that the proposed chaotic dung beetle optimization-boosted multi-agent deep reinforcement learning model achieves superior performance compared to baseline algorithms. Specifically, it reaches an accuracy of over 97.00% with rapid convergence, reduces system cost under varying health data sizes and Medical Internet of Things devices, and maintains robust scalability across diverse workloads. Moreover, the model achieves significant reductions in communication latency and energy consumption as central processing unit cycles and bandwidth increase, while effectively prioritizing high-criticality tasks.

Due to the complexity of three-dimensional (3D) human pose, it is difficult for ordinary sensors to capture subtle changes in pose, resulting in a decrease in the accuracy of 3D human pose detection. A novel 3D human motion pose detection method is designed by combining Nano sensors and multi-agent deep reinforcement learning technology. First, Nano sensors are placed in key parts of the human to collect human electromyogram (EMG) signals. Second, after de-noising the EMG signal by blind source separation technology, the time-domain and frequency-domain features of the surface EMG signal are extracted. Finally, in the multi-agent environment, the deep reinforcement learning network is introduced to build the multi-agent deep reinforcement learning pose detection model, and the 3D local pose of the human is output according to the features of the EMG signal. The fusion and pose calculation of the multi-sensor pose detection results are performed to obtain the 3D human pose detection results. The results show that the proposed method has high accuracy for detecting various human poses, and the accuracy, precision, recall and specificity of 3D human pose detection results are 0.97, 0.98, 0.95 and 0.98, respectively. Compared with other methods, the detection results in this paper are more accurate, and can be widely used in medicine, film, sports and other fields.

Resource-constrained project scheduling (RCPS) aims to minimize project duration under limited resource availabilities. The heuristic methods that are often used to solve the RCPS problem make use of different priority rules. The comparative merits of different priority rules have not been discussed in the literature in sufficient detail. This study is a response to this research gap. It compares 17 heuristic priority rules and seeks the best performing heuristic priority rule. This is the first study ever that compares heuristic priority rules by considering combinations of variations in (1) resource allocation procedures, (2) number of activities, (3) number of resource constraints, and (4) resource supply levels. The objective is to understand the relative merits of heuristic rules used in solving the RCPS problem. The findings indicate that the “minimum late finish time” rule generates the shortest predicted project duration when used in parallel resource allocation, whereas the “minimum late start time”, “minimum late finish time”, and the “highest rank of positional weight 2” rules perform best in serial resource allocation. It was also found that parallel resource allocation is slightly superior to serial resource allocation in most instances.

The rise of the artificial intelligence (AI) brings golden opportunity to accelerate the development of the intelligent transportation system (ITS). The platoon control of connected autonomous vehicle (CAV) as the key technology exhibits superior for improving traffic system. However, there still exist some challenges in multi-objective platoon control and multi-agent interaction. Therefore, this paper proposed a connected autonomous vehicle latoon control approach with multi-agent deep reinforcement learning (MADRL). Finally, the results in stochastic mixed traffic flow based on SUMO (simulation of urban mobility) platform demonstrate that the proposed method is feasible, effective and advanced.KeywordsIntelligent transportation systemConnected autonomous vehicleMulti-objective platoon controlMulti-agent deep reinforcement learning

Mobile edge computing (MEC) has emerged as a key enabler for low-latency and computation-intensive applications. However, conventional terrestrial MEC networks face challenges in coverage, scalability, and dynamic environments. To address these limitations, this paper explores unmanned aerial vehicle (UAV)-assisted MEC, leveraging the mobility and flexibility of UAVs to enhance edge computing performance. However, challenges such as dynamic mobile user association, multi-UAV collaborative trajectory design, and limited communication and computation resources remain significant hurdles. We then formulate a joint optimization problem for user association, trajectory design, and resource allocation for the multi-UAV collaborative MEC network, aiming to minimize system cost considering computation task demands and UAV resource constraints. The problem, modeled as a Mixed Integer NonLinear Program (MINLP), is reformulated as a decentralized partially observable Markov decision process (Dec-POMDP) to capture complex system dynamics. Afterwards, we propose a multi-agent generative-enhanced deep reinforcement learning method that integrates generative diffusion model (GDM) with multi-agent deep reinforcement learning (MADRL) to effectively handle high-dimensional action spaces and enable collaborative multi-UAV decision-making. Extensive simulations using a real-world mobility dataset demonstrate that the proposed method consistently outperforms three baselines in terms of time-average system cost, task completion delay, task satisfactory ratio, and energy consumption ratio when varying the number of UAVs.

With the emergence and development of 5G technology, Mobile Edge Computing (MEC) has been closely integrated with Internet of Vehicles (IoV) technology, which can effectively support and improve network performance in IoV. However, the high-speed mobility of vehicles and diversity of communication quality make computing task offloading strategies more complex. To solve the problem, this paper proposes a computing resource allocation scheme based on deep reinforcement learning network for mobile edge computing scenarios in IoV. Firstly, the task resource allocation model for IoV in corresponding edge computing scenario is determined regarding the computing capacity of service nodes and vehicle moving speed as constraints. Besides, the mathematical model for task offloading and resource allocation is established with the minimum total computing cost as objective function. Then, deep Q-learning network based on deep reinforcement learning network is proposed to solve the mathematical model of resource allocation. Moreover, experience replay method is used to solve the instability of nonlinear approximate function neural network, which can avoid falling into dimension disaster and ensure the low-overhead and low-latency operation requirements of resource allocation. Finally, simulation results show that proposed scheme can effectively allocate the computing resources of IoV in edge computing environment. When the number of user uploaded data is 10K bits and the number of terminals is 15, it still shows the excellent network performance of low-overhead and low-latency.

Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

The dynamic dispatching of mobile charging vehicles (MCVs) has emerged as a promising solution to overcome the limitations of fixed charging infrastructure in electric vehicle (EV) services. However, the stochastic nature of user charging demands and the high operational costs of MCV travel pose significant challenges in balancing supply and demand, particularly under real-time requests and large-scale traffic networks. Existing MCV dispatch strategies are often static, reliant on predefined demand locations, and fail to adapt to the spatiotemporal variations in EV demand and MCV states. To address these challenges, we formulate the MCV dispatching problem as a Markov decision process (MDP) and propose multiagent deep reinforcement learning (MARL)-evolutionary strategy (ES), a novel MARL framework with centralized training and decentralized execution (CTDE), enhanced by integrated action-space ESs. MARL-ES leverages a neural network-based state-value function that incorporates the experiences of MCVs and EV demands, as well as offline historical data. It further employs mutation and segment-based crossover operators to enhance the strategic diversity, improve exploration, and optimize dispatch decisions. We develop a realistic, large-scale city grid simulator to model stochastic EV arrivals, energy constraints, and coordinated MCV interactions. Experimental results show that MARL-ES significantly outperforms static optimization and conventional MARL approaches in total profit, operational cost, and MCV moving distance. The framework also demonstrates robust scalability and adaptability across different fleet sizes and under uncertain demand and energy conditions, offering a practical and adaptive dispatching solution for intelligent mobile charging services (MCSs).

In green transportation systems, Vehicular Edge Computing (VEC) is playing a critical role by enabling real-time responses to compute-intensive tasks. Furthermore, incentivizing diverse stakeholders in VEC to share resources and ensuring equitable resource allocation are pivotal to achieving sustainability. In existing studies, a key limitation is that resource price is often simplified to a static decision factor, thus failing to function as a dynamic indicator that reflects real-time load state of the RSUs and the essential balance between the total utility of RSUs and vehicle user experience (QoE). At the same time, the issue is more challenging when facing the load balancing within multi-RSU. To resolve this issue, the paper builds a differentiated resource price model and designs a pricing strategy that combines vehicle QoE and RSUs cost, in order to maximize the total utility of RSUs. Specifically, the paper propose an Incentive-Driven Collaborative Resource Management (IDCRM) algorithm, which integrates the Soft Actor-Critic (SAC) based Computing Resource Allocation and Pricing (SCRAP) algorithm for dynamic resource allocation and pricing by individual RSU, and RSUs cooperative offloading algorithm to address multi-RSU load balancing. The experimental results demonstrate that our proposed IDCRM effectively maximizes the provider’s revenue while ensuring vehicle QoE and significantly alleviates the resource bottleneck of a single RSU through a collaborative mechanism, achieving efficient load balancing across multi-RSU.

The spectrum sharing design (such as the underlay sidelink configuration in cellular networks) for vehicle-to-vehicle (V2V) and vehicle-to-network (V2N) links can improve the efficiency of spectrum utilization for vehicular networks. However, differentiated service requirements for V2V and V2N links and the greater complexity of interference management make it challenging for conventional centralized learning-based methods in the face of real-world deployment. This motivates the exploration of multi-agent deep reinforcement learning (DRL) solutions, while existing research has predominantly focused on cooperative setting that causes large communication overheads for real-world deployment. This paper proposed a decentralized resource allocation scheme for joint spectrum sub-band and V2V transmission power selection that utilizes federated learning (FL) to formulate a mixed setup to avoid frequent communication and enable easy model sharing, termed FL-MX for the design of FL and mixed DRL agent cooperation level. Different levels of cooperation among agents are also explored by implementing different state-of-the-art (SOTA) DRL-based resource allocation algorithms on the developed standard-compliant simulation platform. Through extensive simulations, it is observed that the proposed FL-MX manages to converge effectively and can match the best SOTA performance, gaining a 22.55% and 9.5% improvement in the V2V payload delivery ratio and V2N throughput compared to the random baseline. Such results make it promising for future research to investigate multi-agent DRL designs at a mixed agent cooperation level for high-performing network decision solutions with less frequent communication demands.

The efficient deployment and fine-tuning of foundation models are pivotal in contemporary artificial intelligence. In this study, we present a groundbreaking paradigm inte-grating 6G Mobile Edge Computing (MEC) with foundation models, specifically designed to enhance local task performance on user equipment (UE). Central to our approach is the innovative Emulator-Adapter architecture, segmenting the foundation model into two cohesive modules. This design not only conserves computational resources but also ensures adaptability and fine-tuning efficiency for downstream tasks. Additionally, we introduce an advanced resource allocation mechanism that is fine-tuned to the needs of the Emulator-Adapter structure in decentralized settings. To address the challenges presented by this system, we employ a hybrid multi-agent Deep Reinforcement Learning strategy, adept at handling mixed discrete-continuous action spaces, ensuring dynamic and optimal resource allocations. Our comprehensive simulations and validations underscore the practical viability of our approach, demonstrating its robustness, efficiency, and scalability. Collectively, this work offers a fresh perspective on deploying foundation models and balancing computational efficiency with task proficiency.

Edge Computing (EC) has been proposed as a promising approach to fulfil the requirements of high bandwidth and ultra-latency of mobile applications. However, existing researches on resource allocation only consider the computing resource of mobile devices and EC servers, while ignored the constraint of the networking resources once spreading applications among multiple EC servers via wireless and wired networks. Fibre-Wireless access networks (FiWi) combine the huge bandwidth of optical fibre networks and flexible access of wireless networks to address the above issue and bridge the coexistence of multiple EC servers. They propose a Virtualisation-based Architecture converging EC over FiWi (VAECFW) to centralise control and allocate networking and computing resources for serving requested services. In addition, they study the problem of resource allocation of EC over FiWi and propose two algorithms Revenue-based Virtual Network Embedding (R-VNE) and Balanced Central Processing Unit Resource Allocation with Virtual Network Embedding (BCRA-VNE). Simulation experiments show that the services acceptance ratio is increased about 35% under their proposed architecture, and the average service requests bandwidth utilisation of R-VNE is increasing from 50 to 66%, and the BCRA-VNE is from 37 to 48%. The two algorithms not only achieve higher revenue but also get better profit rate.

Managing large-scale, geographically distributed, and long-term risks arising from diverse underlying causes – ranging from poverty to underinvestment in protecting against natural hazards or failures of sociotechnical, economic, and financial systems – poses formidable challenges for any theory of effective social decision-making. Participants may have different and rapidly evolving local information and goals, perceive different opportunities and urgencies for actions, and be differently aware of how their actions affect each other through side effects and externalities. Six decades ago, political economist Charles Lindblom viewed “rational-comprehensive decision-making” as utterly impracticable for such realistically complex situations. Instead, he advocated incremental learning and improvement, or “muddling through,” as both a positive and a normative theory of bureaucratic decision-making when costs and benefits are highly uncertain. But sparse, delayed, uncertain, and incomplete feedback undermines the effectiveness of collective learning while muddling through, even if all participant incentives are aligned; it is no panacea. We consider how recent insights from machine learning – especially, deep multiagent reinforcement learning – formalize aspects of muddling through and suggest principles for improving human organizational decision-making. Deep learning principles adapted for human use can not only help participants in different levels of government or control hierarchies manage some large-scale distributed risks, but also show how rational-comprehensive decision analysis and incremental learning and improvement can be reconciled and synthesized.