Radio Resource Allocation Method for Network Slicing using Deep Reinforcement Learning

In the fifth-generation of mobile communications, network slicing is used to provide an optimal network for various services as a slice. In this paper, we propose a radio access network (RAN) slicing method that flexibly allocates RAN resources using deep reinforcement learning (DRL). In RANs, the number of slices controlled by a base station fluctuates in terms of user ingress and egress from the base station coverage area and service switching on the respective sets of user equipment. Therefore, when resource allocation depends on the number of slices, resources cannot be allocated when the number of slices changes. We consider a method that makes optimal-resource allocation independent of the number of slices. Resource allocation is optimized using DRL, which learns the best action for a state through trial and error. To achieve independence from the number of slices, we show a design for a model that manages resources on a one-slice-by-one-agent basis using Ape-X, which is a DRL method. In Ape-X, because agents can be employed in parallel, models that learn various environments can be generated through trial and error of multiple environments. In addition, we design a model that satisfies the slicing requirements without over-allocating resources. Based on this design, it is possible to optimally allocate resources independently of the number of slices by changing the number of agents. In the evaluation, we test multiple scenarios and show that the mean satisfaction of the slice requirements is approximately 97%.

Radio Access Network (RAN) slicing enables resource sharing among multiple tenants and is an essential feature for next-generation mobile networks. Usually, a centralized controller aggregates available resource pools from multiple tenants to increase spectrum availability. In dynamic resource allocation, a tenant could behave strategically by adjusting its preferences based on perceived conditions to maximize its utility. Slice tenants may lie about the resources needed to gain greater utility. Such behavior could lead to poor resource utilization due to excess resources acquired by lying tenants and resource shortages because slice tenants choose not to purchase high-priced resources to save costs. Furthermore, in a scenario with many slice tenants, the centralized controller can become overwhelmed by the number of requests. This, in turn, can lead to slower response times and higher latency, resulting in poor resource utilization and QoS performance of slice tenants. Therefore, this paper proposes a peer-to-peer (P2P) approach to resource trading, where slice tenants communicate directly instead of relying on a centralized orchestrator. This design is motivated by the need for slice tenants to collaborate effectively. We model the interaction between tenants in a Stackelberg multi-leader and multi-follower game and solve the game with multi-agent deep reinforcement learning with an incentive-reward model to achieve the Stackelberg equilibrium. Furthermore, we propose a decentralized resource trading framework by integrating blockchain technology and federated deep reinforcement learning, enabling network tenants to perform inter-slice resource sharing securely. The simulation results show that the proposed mechanism has significant performance improvements over existing implementations.

5G radio access network (RAN) slicing is gaining momentum in finding applications in a wide range of domains, especially those requiring high-speed data transmissions such as space-air-ground integrated networks (i.e., drone-based systems). A key challenge in building a robust RAN slicing to support these applications is, therefore, designing a RAN slicing (RS)-configuration scheme that can utilize information such as resource availability in substrate networks as well as the interdependent relationships among slices to map (embed) virtual network functions (VNFs) onto live substrate nodes. With such motivation, we propose a machine-learning-powered RAN slicing scheme that aims to accommodate maximum numbers of slices (a set of connected VNFs) within a given request set. We present a deep reinforcement scheme that is called Deep Allocation Agent (DAA). In short, DAA utilizes an empirically designed deep neural network that observes the current states of the substrate network and the requested slices to schedule the slices of which VNFs are then mapped to substrate nodes using an optimization algorithm. DAA is trained towards the goal of maximizing the number of accommodated slices in the given set by using an explicitly designed reward function. Our experiment study shows that, on average, DAA is able to maintain a rate of successfully routed slices above 80% in a resource-limited substrate network, and about 60% in extreme conditions, i.e., the available resources are much less than the demands.

Network slicing is a fundamental feature of 5G systems that facilitates the provision of particular system behaviours adapted to specific service/application domains on top of a common network infrastructure. A network slice is in general composed by a core network slice and a Radio Access Network (RAN) slice. The realization of RAN slices is particularly challenging because it requires configuring and operating traffic differentiation and protection mechanisms to simultaneously deliver multiple and diverse RAN behaviors over a given pool of radio resources. In this context, this paper proposes to characterize the behavior of a RAN slice through the specification of a set of control parameters that are used to dictate the operation of the packet scheduling function at Layer 2 and the radio admission control function at Layer 3. An evaluation of the suitability of these parameters for achieving efficient radio resource sharing and isolation between RAN slices is presented when configuring a network for supporting a slice with multiple enhanced Mobile BroadBand services and another slice for providing Mission Critical services. The analysis reveals the different impact of the Layer 3 and Layer 2 parameters for isolating services of different slices depending on whether they require guaranteed or non-guaranteed bit rates.

Fifth-generation (5G) networks are already available in major urban areas and are expected to bring a major transformation to citizens’ lives. 5G services, such as enhanced mobile broadband (eMBB), ultra-reliable low latency communications (URLLC), and massive machine-type communications (mMTC), require a network infrastructure capable of supporting stringent requirements in terms of latency and bandwidth demands; as such, it must be highly dynamic and flexible. Network slicing is a key enabler technology that can provide dynamic and flexible characteristics to 5G network architecture. A network slice (NS) can be defined as a partition of network and IT resources, that is, network links and nodes capacity dedicated to a specific set of service demands. As a result, different NSs can coexist over the same physical infrastructure network and can be used to dynamically and flexibly deploy the aforementioned 5G services. However, to efficiently implement NSs with different requirements, communication service providers (CSPs) that own the physical infrastructure network must adopt sophisticated techniques for admission control and resource allocation of NSs. In this paper, we present a novel framework for admission control and resource allocation of 5G NSs in metro-core networks. Specifically, our framework is based on a deep reinforcement learning (DRL) algorithm called Advantage Actor Critic (A2C), which performs admission control, i.e. it is capable of learning which slice to admit based on the availability of the physical network resources. Then, given the diversity of requirements for each 5G service, we propose different resource allocation algorithms based on integer linear programming (ILP) and heuristics to treat each service accordingly. Results show that our proposed framework can increase the number of admitted NSs with respect to the case in which the admission control is disabled by improving the resource allocation performance.

Network slicing is one of the key enabling techniques for 5G, allowing Mobile Network Operators (MNOs) to support services with diverging requirements on top of their infrastructure. The MNOs should be able to offer network slices as a service and provide customisable and independent virtual networks to verticals. The slicing of an end-to-end (E2E) mobile network is divided into Core Network (CN) slicing, and Radio Access Network (RAN) slicing. In this paper, we assess the requirements for using radio hypervisors to enable RAN as a Service (RANaaS). We evaluate the current state-of-the-art on radio virtualisation with respect to these requirements and identify the missing features. Then, we present the eXtensible Virtualisation Layer (XVL), a software layer that provides the missing functionality for enabling RANaaS and can be added on top of existing radio hypervisors. We outline XVL's architecture and design choices, as well as evaluate its performance in terms of the delay to provision virtual radios, the delay introduced to forward IQ samples, and the computational overhead. Our results show that XVL enables leveraging existing radio hypervisors to support RANaaS.

Network slicing is considered as an efficient method to satisfy the distinct requirement of diversified services by one single infrastructure in 5G network. However, owing to the cost of information gathering and processing, it’s hard to swiftly allocate resources according to the changing demands of different slices. In this demo, we consider a radio access network (RAN) scenario and develop several deep reinforcement learning (DRL) algorithms which can keenly catch the varying demands of users from different slices and learn to make an intelligent decision for resource allocation. Besides, in order to implement and evaluate our algorithms efficiently, we have also implemented a platform with a modified 3GPP Release 15 base station and several on-shelf mobile terminals. Numerical analyses of the corresponding results verify the superior performance of our methods.

Multi-access Edge Computing fundamentals, services, enablers and challenges: A complete survey

With the advent of 5G era, network slicing has received a great deal of attention as a means to support a variety of wireless services in a flexible manner. Network slicing is a technique to divide a single physical resource network into multiple slices supporting independent services. In beyond 5G (B5G) systems, the main goal of network slicing is to assign the physical resource blocks (RBs) such that the quality of service (QoS) requirements of eMBB, URLLC, and mMTC services are satisfied. Since the goal of each service category is dearly distinct and the computational burden caused by the increased number of time slots is huge, it is in general very difficult to assign RB properly. In this paper, we propose a deep reinforcement learning (DRL)-based network slicing technique to find out the resource allocation policy maximizing the long-term throughput while satisfying the QoS requirements in the B5G systems. Key ingredient of the proposed technique is to reduce the action space by eliminating undesirable actions that cannot satisfy the QoS requirements. Numerical results demonstrate that the proposed technique is effective in maximizing the long-term throughput and handling the coexistence of use cases in the B5G environments.

As one of key technologies of the fifth-generation (5G) communication system, network slicing can share the underlying infrastructure with different application requirements and ensure that the slices can be isolated from each other. This paper proposes an end-to-end (E2E) network slicing resource allocation algorithm based on Deep Q-Networks (DQN), which is suitable for multi-slice and multi-service scenarios. This algorithm jointly considers the radio access network slices and core network slices to dynamically allocate resources to maximize the number of access users. First we build such a model, which is a mixed integer programming problem and it needs to be dynamically adjusted according to the changes of environment. We propose to use DQN algorithm to solve this problem, which can perceive changes in the environment and make dynamic decisions. Under each decision, we need to calculate the reward value of DQN, so we divide the problem into the core side and the access side. Then the dynamic knapsack algorithm and the link mapping algorithm are used to obtain the reward. The simulation results show that the average access rate of DQN scheme is higher than 97%. Compared with the optimal allocation scheme of access side, the average access rate is increased by 9% for delay constrained slices and 5% for rate constrained slices in a dynamic environment.

In 5G networks, the deployment of network slices enabled by Software-Defined Networking (SDN) is becoming a critical component for delivering tailored services to meet diverse application needs. However, this introduces challenges in network management, particularly in efficiently allocating resources to ensure that each network slice meets its specific Quality of Service (QoS) and availability requirements. Simultaneously, it must optimize overall network performance and network operator’s profit, which is linked to the Quality of Experience (QoE) of the end-users. Existing works offer either an availability-based solution or a QoE-aware solution to this problem, but not both. This paper addresses the end-to-end network slice resource allocation problem by simultaneously considering QoS and availability requirements in slice placement, while employing a QoE-aware strategy for resource allocation. We propose a framework that optimizes the network operator profit i.e. the highest QoE with the least resource usage, and can be flexibly configured to model realistic scenarios. Arbitrary network slice requirements can be defined using slice-specific QoS/QoE mapping, resource requirements, end-to-end latency and availability. For solving the formulated problem a Mixed Integer Nonlinear Programming (MINLP) formulation and efficient heuristic methods are proposed. Our solution accounts for the non-linear QoS/QoE relationship, utilizes redundantly placed Service Function Chains (SFCs) to increase availability, and supports the sharing of Virtual Network Functions (VNFs) among SFCs to optimize resource usage. Through extensive simulations on realistic network topologies and slice requests, we demonstrate the framework’s effectiveness in offering flexible and efficient network slice placement and resource allocation, utilizing a baseline heuristic from related studies. The results indicate that while the exact method delivers an optimal solution, heuristic approaches are suitable for time-sensitive tasks, such as dynamic slice configuration.

The deployment of reliable maritime communication systems integrated with terrestrial networks is challenging due to the large maritime regions, the difficulty of deploying conventional base stations at sea, and the heterogeneity of proprietary equipment. In this paper, we propose an AI-based network slicing framework for Open Radio Access Network (O-RAN) integrated aerial-terrestrial maritime networks that incorporates non-tethered and tethered unmanned aerial vehicles (UAVs) and marine buoys. The network provides ubiquitous connectivity while addressing diverse maritime user requirements. Specifically, we leverage network slicing to accommodate the needs of two distinct slices: the maritime infotainment slice, which demands high data rates, and the maritime emergency communication slice, which requires high-reliability and low-latency. Moreover, we adopt virtualization principles to enable flexible deployment approaches for virtualized network functions (VNFs), that can be dynamically scaled and/or migrated across virtualized network nodes. Then, we design a network slicing framework based on a Deep Reinforcement Learning (DRL) that takes into account the characteristics of the maritime environment, and present two algorithms using Advantage Actor-Critic (A2C) and Proximal Policy Optimization (PPO). Our findings highlight the importance of the integration of aerial and terrestrial networks with network slicing to enhance the energy efficiency of maritime communications while meeting diverse Quality of Service (QoS) requirements.

Network slicing is included as a key feature of the 5G architecture in order to simultaneously support diverse service types with heterogeneous requirements. The deployment of network slicing in the Radio Access Network (RAN) needs mechanisms that allow the distribution of the available capacity in the system in an efficient manner while satisfying the requirements of the different services. In this paper, a capacity sharing function is proposed, which is approached as a multi-agent reinforcement learning based on the Deep Reinforcement Learning (DRL) algorithm Deep Q-Network (DQN). The proposed algorithm provides the capacity to be assigned to each RAN slice. Performance assessment reveals the promising behaviour of the proposed solution.

Network slicing in 5G and beyond networks allows the network to be customized for each application or service by chaining together different virtualized network functions (VNFs) according to service requirements. The increased flexibility offered by network slicing comes at the cost of complexity in management and orchestration, which cannot be solved by traditional reactive human-in-the-loop solutions. This necessitates minimizing human intervention through the use of artificial intelligence techniques (zero touch network management). In particular, the scaling and placement of the chain of VNFs that constitute a network slice is a complex combinatorial optimization problem that is difficult to solve effectively with traditional approaches. Driven by the benefits of deep reinforcement learning (DRL) in solving various combinatorial optimization problems, in this article, we survey various DRL-based approaches to slice scaling and placement, including different ways to model the problem and benefits of various DRL techniques in addressing specific aspects of the problem. Further, we highlight key challenges and open issues in the effective use of DRL for network slice scaling and placement.

The fifth-generation mobile network presents a wide range of services which have different requirements in terms of performance, bandwidth, reliability, and latency. The legacy networks are not capable to handle these diverse services with the same physical infrastructure. In this way, network virtualization presents a reliable solution named network slicing that supports service heterogeneity and provides differentiated resources to each service. Network slicing enables network operators to create multiple logical networks over a common physical infrastructure. In this research article, we have designed and implemented an intent-based network slicing system that can slice and manage the core network and radio access network (RAN) resources efficiently. It is an automated system, where users just need to provide higher-level network configurations in the form of intents/contracts for a network slice, and in return, our system deploys and configures the requested resources accordingly. Further, our system grants the automation of the network configurations process and reduces the manual effort. It has an intent-based networking (IBN) tool which can control, manage, and monitor the network slice resources properly. Moreover, a deep learning model, the generative adversarial neural network (GAN), has been used for the management of network resources. Several tests have been carried out with our system by creating three slices, which shows better performance in terms of bandwidth and latency.