Evaluating Reinforcement Learning-Based Xapps for User-Centric Resource Allocation in Open RAN

Abstract

Translate Article

Dynamic radio resource management is essential for optimising performance in an open radio access network (RAN). However, traditional methods often struggle with realtime decision-making and resource allocation in dynamic environments. Intelligent solutions, such as xApps running on near-real-time (near-RT) RAN Intelligent Controllers (RICs), offer promising advancements in adaptability and efficiency. This paper evaluates the performance of 5 reinforcement learning (RL) algorithms: Advantage Actor-Critic (A2C), Asynchronous Advantage Actor-Critic (A3C), Proximal Policy Optimisation (PPO), Deep Q-Network (DQN) and Soft Actor Critic (SAC), which were implemented as xApps for dynamic resource allocation in an open RAN setting. We analyse how effectively each algorithm learns to allocate resources by associating user equipment (UE), connected to different service classes: video, voice and data, to base stations as they move through a simulated network environment. Additionally, we evaluate the agents by comparing their throughput, latency and reward performance, and validate the results by examining them over 5 seeds run under the same network configurations. Our findings reveal that DQN achieves the highest rewards and fastest convergence, followed by SAC, PPO offers stable learning at moderate reward levels, while A2C and A3C show slower convergence and lower rewards. These results demonstrate both the advantages and limitations of using RL for dynamic resource management in open RAN systems. Finally, we discuss future directions, such as exploring multiagent systems and deploying xApps in real-world open RAN testbeds, to advance practical applications.