ExpertDRL: Request Dispatching and Instance Configuration for Serverless Edge Inference With Foundation Models

Pretraining geospatial foundation models (FMs) is expensive, and architectural choices control inductive bias for multiscale context, cross-resolution behavior, and band/sensor variation. Therefore, benchmarking reduces the risk of scaling the “wrong” base. A model benchmarking framework for geospatial image segmentation is a critical prerequisite for developing robust and scalable geospatial FMs. In the emerging era of Earth observation FMs, success hinges on strong, well-characterized base architectures that can generalize across sensors, modalities, and geographies. The extreme heterogeneity of Earth observation vision data (different spectral bands, resolutions, and regions) makes such generalization especially challenging, underscoring the need for systematic, controlled benchmarking across diverse model families to identify viable architectures for different scenarios.Our work rigorously evaluates a broad spectrum of segmentation architectures and backbones under consistent conditions. We benchmark classical convolutional architectures (U-Net, DeepLab, UPerNet, FPN, PAN, and LinkNet) alongside modern transformer-based models (Dense Prediction Transformer (DPT) and SegFormer). For this comparison, we use representative backbones from both CNNs (ResNet and MobileNet) and Mix Vision Transformer (MiT). By comparing these heterogeneous models on equal footing, we determine which architectural patterns and hybrid combinations yield representations most conducive to generalization. This diversity in evaluation identifies well-founded architectural bases for geospatial FMs.To guide architecture selection and pipeline design, we deploy a comprehensive suite of metrics covering both accuracy and efficiency. We evaluate segmentation accuracy via IoU, Dice, and boundary F1-score, and also measure efficiency (convergence speed and inference latency). These holistic benchmarks reveal critical trade-offs. For instance, some lightweight CNN models excel in speed, while transformer models boost boundary F-1 score. By capturing such nuances, our benchmark informs which architectures are best suited as general-purpose base models. It highlights how certain encoder–decoder combinations optimally balance performance and efficiency, and flags architectures with high transfer-readiness for new tasks and domains.The result is a reproducible, transferable model landscape that serves as a blueprint for FM development. Our benchmark framework effectively preconditions the FM pipeline, enabling researchers to enter the scaling phase with proven architecture candidates that have demonstrated cross-task and cross-sensor robustness. This “model landscape” allows subsequent large-scale pretraining to confidently build on architectures that ensure broad downstream generalization even in agentic (autonomous) deployment scenarios.Finally, we situate this work within the broader trend toward sensor-agnostic, self-supervised FMs in Earth observation. We argue that intelligent architecture search must precede any massive self-supervised pretraining effort. Early vetting of architectures under diverse conditions ensures that large-scale training resources are invested in the most promising designs. In summary, we frame this hybrid benchmarking framework as a strategic new layer in the geospatial FM ecosystem. The insights extend beyond segmentation, providing a reference point for building fine-tunable, sensor-agnostic foundation models that can be readily adapted to various downstream tasks and even deployed onboard satellites or other edge platforms. By solidifying architecture evaluation as an essential step, this work makes a serious scientific and strategic contribution toward the next generation of Earth observation AI.

Introduction: Tumor single-cell data provide unique insights into the cellular dynamics of cancer progression and therapeutic outcomes. Inspired by advancements in natural language processing (NLP), foundation models are increasingly used in single-cell analyses to uncover biological insights and identify clinical biomarkers. However, the performance of these models on cancer-specific tasks remains underexplored. We present an extensive evaluation of foundation models trained on millions of single-cell profiles for cancer-focused investigations. Methods: We evaluated nine predictive models-three traditional methods and six foundation models (four Geneformer and two scGPT models), across various downstream tasks. Traditional models used embeddings from high variable genes (HVG), principal components (PCA), and generative model embedding (scVI), each paired with a random forest classifier for downstream classification. The evaluated foundation models varied in architecture sizes, pretraining dataset sizes (30M to 95M cell profiles), and training datasets, which were either generic or cancer-specific. We assessed the quality of representations by measuring their predictive performance on lung cancer treatment status (TKI treated vs. treatment-naive patients) and breast cancer subtypes (ER+ vs. TNBC). We examined how model characteristics influenced their performance in these domains for all tasks. Results: Zero-shot evaluations of the foundation models showed superior single-cell data clustering, measured by average silhouette width (ASW), compared to traditional methods (HVG, PCA, scVI), suggesting that pre-trained embeddings from millions of single-cell profiles may offer enhanced representations. We thus evaluated the learned representation on downstream predictive tasks, showing that foundation models with a larger pretraining dataset (95M cells), cancer-specific training, larger input size (4096 genes), and deeper model (12 layers) achieved the best performance among foundation models (F1=0.97) for predicting treatment status in lung cancer. In contrast, smaller and generic foundation models had significantly lower performance, however, additional self-supervised training on task-specific data rescued their performance to match larger foundation models. Similar trends were observed in predicting breast cancer subtypes. Notably, the best foundation model’s performance was not significantly different from the best traditional method (PCA) in predicting breast cancer subtypes. Conclusions: These results highlight key factors driving foundation model performance on cancer-specific tasks. While current foundation models show inconsistent contributions to cancer-specific questions, observed performance trends highlight their potential to enhance single-cell data representation and drive significant advancements in cancer research and precision medicine. Citation Format: Haitham Elmarakeby, Ahmed Roman, Shreya Johri, Eliezer M. Van Allen. Evaluation of single-cell foundation models for cancer outcome predictions [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5084.

Foundation models (FMs) have revolutionized generative AI (GAI) lifecycle with their pre-trained intelligence capabilities. While the recent success of web-based models like GPT-4 has spurred interest in extending FMs to edge scenarios like the Internet of Vehicles (IoV), challenges such as data privacy, network congestion, and limited edge resource awareness hinder the direct application of cloud-based FMs. To address these challenges, this paper reconfigures the mixture of experts (MoE) architecture to deploy Transformer-based FM experts across vehicles for distributed GAI inference. Specifically, we propose an MoE-empowered vehicular FM system with two key innovations: a physical gating network that dynamically adapts to wireless environments, and a vehicle-to-vehicle (V2V) communication-based expert parallelism mechanism to enhance efficiency and resource utilization. Then, we formulate the communication, computation, and memory models for analyzing inference latency. Furthermore, to optimize the computational resource allocation for enhancing inference performance, we establish a Stackelberg game and propose the Gradient Ascent and Evolutionary Optimization-based Competitive Pricing and Allocation (GECPA) algorithm, which balances resource allocation and usage costs by combining the rapid convergence of gradient ascent with the broader exploration capability of evolutionary optimization. Simulation results demonstrate the superior parallel processing efficiency and reduced latency of the proposed MoE-based FM inference scheme. Compared with the best-performing benchmark algorithm, GECPA improves the average utility of infrastructure vehicles by up to 2.01 times, reduces inference latency by 16.58%, and increases the successful execution rate of GAI tasks by 10.66%, thereby achieving a better balance between efficiency and incentive compatibility in dynamic IoV environments.

Attempts to address the production scheduling problem thus far rely on simplifying assumptions, such as static environment and inflexible size of the problem, which compromises the schedule performance in practice due to many unpredictable disruptions to the system. Thus, the study of scheduling in the presence of real-time events, termed dynamic scheduling, continues to attract attention given the agility, flexibility, and timeliness modern production systems must deliver. Additionally, the changing nature of the manufacturing system also raises new challenges to existing scheduling strategies. At the front-end, the development of advanced data creation and exchange frameworks such as the Internet of things and cyber-physical system and their applications to the industrial environment have created an abundance of industrial data, while at the backend, edge and cloud computing technologies greatly enhance the capacity to process that data. Industrial data must be mined and analyzed so that the investment in infrastructure is not wasted, and the production system managed more effectively and in real-time. Many data-driven technologies have been adopted in scheduling research, a promising candidate among them being reinforcement learning (RL) which is able to build a direct mapping from observation of environment to actions that improve its performance. In this thesis, a deep multi-agent reinforcement learning (deep MARL) architecture is proposed to solve the dynamic scheduling problem (DSP). The deep reinforcement learning (DRL) algorithm is used to train the decentralized scheduling agents, to capture the relationship between information on the factory floor and scheduling objectives, with the aim of making real-time decisions for a manufacturing system with frequent unexpected events. Two major aspects of deep MARL application to DSP are addressed in this work, namely the conversion from traditional static scheduling problem (SSP) to dynamic scheduling in a practical context, and the adaptation of existing deep MARL algorithms to solve the scheduling problem in such an environment. Some impractical constraints of traditional studies are removed to create a research context that is closer to actual practice, result in a scheduling problem of variable size and scope. Specialized state and action representations that can handle the ever-changing specification of problem are developed; the criteria of feature selection in dynamic environment are also discussed. Recent progressions in DRL and MARL research are integrated into the proposed approach after selection and adaptation. In addition, various improvements to common deep MARL architecture are proposed, including the lightweight multilayer perceptron (MLP) encoder that is efficient in handling unstructured industrial data, a training scheme under the multi-agent architecture to improve the stability of training and overall performance, and knowledge-based reward-shaping techniques to decompose the joint reward signal into individual utilities to speed up the learning and encourage cooperative behavior between agents. Simulation studies are then conducted for the ablation study and validation. In the first stage, the performance of the proposed approach, either as individual components or as an integrated model, are tested in iterative simulation runs within which a unique instance of production is created. Meanwhile, a set of DRL-based approaches from recent publications are run in parallel. Results suggest that the contribution of each improvement is significant; the integrated architecture also delivers stronger performance than peer DRL-based approaches. For the validation, a set of priority rules that have strong performance in specified context and are widely applied in actual production scheduling are used as the benchmark. Proposed approach also provides performance gain compared to the strongest rule, with a minor increase in computation cost and negligible latency in decision-making.

Irrigation decision-making using Reinforcement Learning (RL) performs well in changing environment, but easily falls into sub-optimal solutions with high-dimensional data. Deep Reinforcement Learning (DRL) has fused RL with Deep Learning (DL) and excels at learning adaptive and long-term irrigation strategies directly from high-dimensional environment data. This paper systematically reviews the applications of DRL in irrigation optimization, covering both pre-trained environments based on crop growth simulators and dynamic environments driven by real-time sensors. We discussed the strengths of classic DRL algorithms, including their ability to handle dynamic and non-linear environments, and reviewed their performance in irrigation multi-objective optimization and decision-making. In addition, we identified constraints in applying DRL in irrigation decision making, which include data scarcity, poor model interpretability, and difficulties in field deployment. It shows DRL can provide a powerful framework for adaptive irrigation, but is constrained by the gap between simulation and real-world complexity. To address these limitations, we discussed approaches in future work, such as developing multi-objective DRL algorithms. These approaches will improve DRL modeling outcomes and provide a technological foundation for smart agriculture and sustainable resource management. • Review the application of Deep Reinforcement Learning (DRL) in agricultural irrigation. • Analyze the performance of DRL algorithms in irrigation decision-making. • Compare DRL models based on different environment in irrigation optimization. • Discuss the further work to improve DRL performance in irrigation optimization.

<p><b>The client-centric multi-cloud has become a popular cloud ecosystem because it allows enterprise users to share the workload across multiple cloud providers to achieve high-quality services with lower operation costs and higher application resilience. From the perspective of application providers, the location of cloud resources for application deployment significantly impacts the deployment cost and performance of applications, e.g., request response time. This gives rise to the problem of location-aware application deployment in multi-cloud to select suitable cloud resources from widely distributed multi-cloud data centers to balance the cost and performance. Existing research works did not pay full attention to the key impact of the location for application deployment. Therefore, it is urgent to study the problem both theoretically and in practice. In this thesis, innovative optimization methods and machine learning techniques are proposed for three common scenarios, namely composite application deployment, application replication and deployment, and elastic application deployment.</b></p> <p>First, this thesis studies the composite application deployment problem with the goal to minimize the average response time of composite applications subject to a budget constraint. We propose a Hybrid Genetic Algorithm (GA)-based approach, i.e., H-GA, for solving the problem with an extremely large search space. H-GA features a newly designed and domain-tailored service clustering algorithm, repair algorithm, solution representation, population initialization, and genetic operators. Experiments show that H-GA can outperform significantly several state-of-the-art approaches, achieving up to about 8% performance improvement in terms of response time, and 100% budget satisfaction in the meantime.</p> <p>Second, this thesis studies the application replication and deployment problem with the goal to minimize the total deployment cost of all application replicas subject to a stringent requirement on average response time. We propose two approaches under different optimization frameworks to solve the problem. With user requests dispatched to the closest application replicas, we develop an approach under a GA framework for Application Replication and Deployment (ARD), i.e., GA-ARD. GAARD features problem-specific solution representation, fitness measurement, and population initialization, which are effective to optimize the deployment of application replicas in multi-cloud. The experiments show that GA-ARD outperforms common application replication and placement strategies in the industry. With user requests flexibly dispatched among different application replicas, we develop another approach under a two-stage optimization framework, i.e., MCApp. MCApp can optimize both replica deployment and request dispatching by combining mixed-integer linear programming with domain-tailored large neighborhood search. Our experiments show that MCApp can achieve up to 25% reduction in total deployment cost compared with several recently developed approaches.</p> <p>Third, this thesis studies the elastic application deployment problem to minimize the deployment cost over a time span such as a billing day while satisfying the constraint on average response time. The goal of adapting resources for application deployment in response to dynamic and distributed workloads motivates us to adopt deep reinforcement learning (DRL) techniques. The proposed approach, namely DeepScale, applies a deep Q-network (DQN) to capture the optimal scaling policy that can perform online resource scaling. DeepScale also includes a long short-term memory-based prediction model to allow the DQN to consider predicted future requests while making cost-effective scaling decisions. Besides, we design a penalty-based reward function and a safety-aware action executor to ensure that any scaling decisions made by DRL can satisfy the response time constraint. The experiments show that DeepScale can significantly reduce the deployment cost of applications compared with the state-of-the-art baselines, including Amazon auto-scaling service and recently proposed RL-based algorithms. In the meanwhile, DeepScale can effectively satisfy the constraint on the average response time.</p> <p>In summary, this thesis studies three new problems for location-aware application deployment in multi-cloud. We propose four novel approaches under different optimization and machine learning frameworks, i.e., HGA, GA-ARD, MCApp, and DeepScale, for solving these problems. New constraint handling techniques are developed to satisfy the practical deployment requirements of enterprise applications.</p>

The rapid integration of Artificial Intelligence (AI) and Internet of Things (IoT) technologies has led to the pro liferation of AIoT applications, significantly escalating demands for computing and communication resources in multi-user, multitask scenarios. A critical challenge lies in efficiently managing resource allocation to ensure Quality of Service (QoS) for diverse Deep Neural Network (DNN) inference tasks. Existing edge cloud collaborative inference approaches partially address this by hierarchical resource management; however, these methods often overlook the joint optimization of computation, communication, and data quality, and neglect long-term system stability in dynamic environments. To address these limitations, we propose EdgeManager, an online adaptive resource management frame work for hierarchical DNN inference in collaborative heterogeneous edge environments. Specifically, we formulate a Mixed Integer Nonlinear Programming (MINLP) optimization problem aimed at balancing inference accuracy and latency. Leveraging Lyapunov optimization, we transform the complex, multi-stage dynamic optimization problem into manageable deterministic sub-problems for each time slot, ensuring long-term stability. Furthermore, we introduce HyDRL-MO, a hybrid approach integrating model-free Deep Reinforcement Learning (DRL) and model-based multi-decision optimization techniques to achieve efficient and stable resource allocation. Extensive experimental evaluations demonstrate that EdgeManager significantly improves system performance, achieving up to 42.08% enhancement in average system benefits compared to state-of-the-art solutions.

This article focuses on achieving efficient and safe navigation for robots in dynamic and unpredictable environments. We propose a hierarchical path planning framework that integrates global path planning with local dynamic obstacle avoidance. This framework aims to quickly plan a collision-free, shortest, and safest path for the robot while adapting the navigation path according to uncertainties in the operating environment. Global path planning is conducted using the improved gray wolf optimization algorithm (IGWO), trajectory tracking is achieved through a pure pursuit control algorithm, and a dynamic switching mechanism based on deep reinforcement learning (DRL) significantly enhances the navigation performance of the robotic system. The effectiveness of this approach has been verified through simulations and experiments. Simulation results indicate that in global path planning, the IGWO algorithm achieves faster convergence compared to algorithms, such as GWO-MP and GWO-CS. The planned path lengths are reduced by approximately 4.01% and 2.27%, respectively, and the fitness values are decreased by 4.78% and 1.9%, demonstrating superior path planning performance. For local dynamic obstacle avoidance, both single-robot and multirobot systems successfully avoided obstacles in multiple experiments. Finally, physical experiments conducted in various complex scenarios show that both single-robot and multirobot systems can effectively execute global planning and respond to unexpected obstacles. These results further demonstrate the method’s wide applicability and robust performance across diverse complex environments.

The increasing volume of traffic has led to severe challenges, including traffic congestion, heightened energy consumption, increased air pollution, and prolonged travel times. Addressing these issues requires innovative approaches for optimizing road network utilization. While Deep Reinforcement Learning (DRL)-based methods have shown remarkable effectiveness in dynamic scenarios like traffic management, their primary focus has been on single-agent setups, limiting their applicability to real-world multi-agent systems. Managing agents and fostering collaboration in a multi-agent reinforcement learning scenario remains a challenging task. This paper introduces a cooperative multi-agent federated reinforcement learning algorithm named FLDQN to address the challenge of agent cooperation by solving travel time minimization challenges in dynamic multi-agent reinforcement learning (MARL) scenarios. FLDQN leverages federated learning to facilitate collaboration and knowledge sharing among intelligent agents, optimizing vehicle routing and reducing congestion in dynamic traffic environments. Using the SUMO simulator, multiple agents equipped with deep Q-learning models interact with their local environments, share model updates via a federated server, and collectively enhance their policies using unique local observations while benefiting from the collective experiences of other agents. Experimental evaluations demonstrate that FLDQN achieves a significant average reduction of over 34.6% in travel time compared to non-cooperative methods while simultaneously lowering the computational overhead through distributed learning. FLDQN underscores the vital impact of agent cooperation and provides an innovative solution for enabling agent cooperation in a multi-agent environment.

Deep Reinforcement Learning (DRL) has emerged as a transformative paradigm in the field of artificial intelligence (AI), offering unprecedented capabilities in decision-making across diverse domains. This article explores the profound impact of DRL on enhancing the decision-making capabilities of AI systems, elucidating its underlying principles, applications, and implications.DRL represents a fusion of deep learning and reinforcement learning, enabling machines to learn complex behaviors and make decisions by interacting with their environment. The utilization of neural networks allows DRL algorithms to handle high-dimensional input spaces, making it well-suited for tasks that involve intricate decision-making processes.One of the key strengths of DRL lies in its ability to address problems with sparse and delayed rewards, common challenges in traditional reinforcement learning. Through a process of trial and error, DRL algorithms can learn optimal decision strategies by navigating through a vast decision space, adapting to dynamic environments, and maximizing cumulative rewards over time.The applications of DRL span various domains, including robotics, finance, healthcare, gaming, and autonomous systems. In robotics, DRL facilitates the development of intelligent agents capable of autonomously navigating complex environments, performing intricate tasks, and adapting to unforeseen circumstances. In finance, DRL is leveraged for portfolio optimization, algorithmic trading, and risk management, demonstrating its potential to revolutionize traditional financial strategies.

Recent studies show that Deep Reinforcement Learning (DRL) models are vulnerable to adversarial attacks, which attack DRL models by adding small perturbations to the observations. However, some attacks assume full availability of the victim model, and some require a huge amount of computation, making them less feasible for real world applications. In this work, we make further explorations of the vulnerabilities of DRL by studying other aspects of attacks on DRL using realistic and efficient attacks. First, we adapt and propose efficient black-box attacks when we do not have access to DRL model parameters. Second, to address the high computational demands of existing attacks, we introduce efficient online sequential attacks that exploit temporal consistency across consecutive steps. Third, we explore the possibility of an attacker perturbing other aspects in the DRL setting, such as the environment dynamics. Finally, to account for imperfections in how an attacker would inject perturbations in the physical world, we devise a method for generating a robust physical perturbations to be printed. The attack is evaluated on a real-world robot under various conditions. We conduct extensive experiments both in simulation such as Atari games, robotics and autonomous driving, and on real-world robotics, to compare the effectiveness of the proposed attacks with baseline approaches. To the best of our knowledge, we are the first to apply adversarial attacks on DRL systems to physical robots.

Study question Can a generalizable computer vision foundation model be trained using self-supervised learning on time-lapse embryo images to perform multiple clinically relevant downstream tasks? Summary answer We developed FEMI, a foundation model trained on eight million time-lapse images, to perform multiple clinical tasks, including blastocyst quality scoring, ploidy prediction, and segmentation. What is known already In vitro fertilization success critically depends on choosing viable embryos, a process hampered by current limited diagnostic tools, high costs, and ethical concerns. In recent years, various medical fields have increasingly explored foundation models using vision transformer architectures. These models, trained in a self-supervised manner on vast unlabeled image datasets, perform various clinically relevant tasks. Once trained on a massive unlabeled dataset, the encoder portion of the foundation model can be extracted and subsequently fine-tuned with a labeled dataset to perform various clinically relevant tasks like classification, regression, and segmentation. Study design, size, duration The foundation model was trained on 8 million Embryoscope® (E-SD) and Embryoscope+® (E+) images spanning clinics in the United States, Canada, and Europe. Datasets contained various additional information, such as ploidy status, blastocyst scores (numerical values from 3 to 14 based on Zhan et al. 2020), and segmentation masks (for trophectoderm [TE], inner cell mass [ICM], and zona pellucida [ZP]) that were used for training downstream tasks. Participants/materials, setting, methods A masked autoencoder architecture is trained on time-lapse images to create a foundation model (FEMI). The encoder from the autoencoder is extracted and fine-tuned on three image-based downstream tasks: ploidy prediction, blastocyst scoring, and embryo component segmentation. The performance of the fine-tuned foundation model is compared to VGG16 architectures, specifically trained for each downstream task. We evaluated performances using the area under the receiver-operating-characteristic (AUROC), mean absolute error (MAE), and mean intersection over union (mIoU). Main results and the role of chance The training dataset for the downstream tasks consisted of image data and labels, including PGT-A ploidy results (euploid or aneuploid), blastocyst scores (3-14), and segmentation masks. No clinical data like maternal age was used in order to assess FEMI’s learning capabilities from only time-lapse images against VGG16 model baselines. Both FEMI and baseline models used the same data splits for consistent comparisons. For ploidy prediction, models trained on a Weill Cornell Medicine (WCM) dataset were validated using WCM, Florida, and Spain data. FEMI achieved a 0.610 ± 0.004 AUROC, outperforming the baseline’s 0.590 ± 0.005. In blastocyst score prediction, FEMI attained a superior MAE of 0.099 ± 0.002 compared to the baseline’s 0.118 ± 0.001. For embryo segmentation (TE, ICM, ZP), a publicly available dataset from Simon Fraser University was split for training and validation. FEMI exceeded the baseline in mIoU for TE segmentation (0.738 ± 0.002 vs. 0.726 ± 0.002) and was comparable in ICM (0.832 ± 0.008 vs. 0.821 ± 0.011) and ZP segmentation (0.779 ± 0.002 vs. 0.778 ± 0.010). Limitations, reasons for caution The current version of FEMI is trained on a subset of our available datasets and can be improved through further training. Moreover, the downstream ploidy and blastocyst score models may be biased by errors in PGT results and the subjectivity of embryologists, respectively. Wider implications of the findings A generalizable foundational model for IVF time-lapse imaging aids clinicians in embryo selection by providing various clinical insights for better decision-making. FEMI, once fully trained, will be publicly accessible to the scientific community allowing for researchers to fine-tune FEMI on their own clinic-specific applications with their own image datasets. Trial registration number not applicable

Cell-free massive MIMO (CF-mMIMO) is a promising approach for future mobile networks, utilizing centralized MIMO processing for densely distributed access points (APs). In CF-mMIMO, to reduce the computational load for signal processing while meeting throughput demands, user equipment (UEs) are served by a selected number of APs. A significant challenge is AP clustering for each UE, particularly in dynamic environments with moving UEs. One approach for optimizing the AP cluster involves deep reinforcement learning (DRL). However, with numerous UEs and APs, the computational load of DRL increases due to the larger model size and higher inference frequency. To address this, we propose an AP clustering method using distributed DRL. The model focuses on determining the AP cluster for every single UE to prevent model size expansion. The per-user models act as distributed actors, enabling parallel inference. Furthermore, to suppress inference frequency, multiple UEs with low mobility are assigned to the same actor, minimizing the number of parallel actors required without compromising throughput. Numerical simulation shows that our proposed method achieves efficient AP clustering that satisfies throughput requirements with reduced computational load in DRL, even in large-scale environments.

The integration of Mobile Edge Computing and container virtualization technologies provides crucial support for low-latency and highly resilient service deployment in Internet of Vehicles （IoV） applications. However, the high mobility of vehicles poses challenges to service continuity, necessitating dynamic adjustment of service deployment locations through container migration. Existing research predominantly focuses on independent service migration while overlooking the complex interdependencies among multiple subtasks in practical applications. In this paper, we investigate the container migration problem for dependency-aware services in IoV environments. We first formulate the problem as a dual-objective optimization problem centered on minimizing both the average service delay and system load imbalance. To address the complex dependencies among containers and the highly dynamic nature of IoV environments, we propose an intelligent migration algorithm named GADM that integrates Graph Attention Networks with Deep Reinforcement Learning. The GADM algorithm leverages Graph Attention Networks to capture critical paths in task dependencies, and combines this with an actor–critic-based Deep Reinforcement Learning framework to achieve adaptive decision-making in dynamic environments. Validation using real-world vehicle trajectory datasets and Alibaba cluster trace datasets demonstrates the effectiveness of the proposed algorithm. Experimental results indicate that compared to other methods, GADM significantly improves system load balancing while reducing average service latency.

As an emerging technology, Integrated Sensing and Communications (ISAC) is expected to be applied to uncrewed aerial vehicle (UAV)-assisted Internet of Things (IoT) systems to sense and acquire information from IoT devices in complex environments. In this paper, we design a UAV-ISAC assisted IoT system, where the UAV senses the IoT nodes in unknown locations and simultaneously forwards the sensing information to the ground base station (BS). The UAV is required to complete the sensing of all IoT nodes before returning to the starting position. However, several challenges arise in designing this system, including unknown IoT node positions, a non-convex optimization problem, constrained resources, a dynamic environment, and complex resource allocation. We propose using radar estimation rate as the measure for evaluating the UAV sensing performance. We formulate a UAV trajectory optimization problem that aims to maximize the radar estimation rate and minimize the UAV flight time, while adhering to constraints on radar sensing performance, information rate, and sensing all the nodes. However, solving this problem using standard optimization techniques is challenging due to the lack of information about the locations of IoT nodes and the non-convex nature of the problem. To address this challenge, we propose an algorithm by adopting multi-step learning Dueling Double Deep Q-learning network (DDQN) based on deep reinforcement learning (DRL) to solve this intractable optimization problem. The simulation results demonstrate that our proposed algorithm achieves stable convergence under different constraint thresholds. Furthermore, we verify the superiority of our algorithm by comparing it with other benchmark algorithms.