Distributed Deep Reinforcement Learning for Functional Split Control in Energy Harvesting Virtualized Small Cells

On the competitive effects of mobile virtual network operators

Customer retention, loyalty, and satisfaction in the German mobile cellular telecommunications market

Joint mode selection and resource allocation for cellular V2X communication using distributed deep reinforcement learning under 5G and beyond networks

With the rapid growth of the wireless network scale and the aggressive development of communication technology, the communication network connection is required to drift to digits in order to ameliorate the network efficiency. Digital twin (DT) is one of the most promising techniques, which promotes the digital transition of communication networks by establishing mappings between virtual models and physical objects. Nevertheless, due to the limitation and heterogeneity of equipment resources, it is a great challenge to provide efficient network resource allocation. To solve this problem, the authors propose a novel network paradigm based on digital twin to build the topology and model of the communication system. Then a distributed deep reinforcement learning (DRL) method is designed to dispose the problem of resource allocation in cellular networks, and an online–offline learning framework is proposed. Firstly, the offline training is carried out in the simulation environment, and the DRL algorithm is applied to train the deep neural network (DNN). Secondly, in the process of online learning, the real data are further utilized to fine-tune the DNN. Numerical results illustrate the superiority of the proposed method in terms of average system capacity. In the case of different user densities, the performance of the proposed algorithm has more advantages than that of benchmark algorithms and has better generalization ability.KeywordsDigital twin (DT)Communication networksResource allocationDeep reinforcement learning (DRL)

Future mobile networks are anticipated to have support to access heterogeneous access technologies. An important question in such environments is the decisions on when to use what access network. These decisions must take the users and mobile network operators’ perspectives into consideration. The operators usually have a focus of always-best-managed networks while the users have an always-best-connected focus. A key enabler of such environments is the use of policies to perform these handover decisions. These policies, which must be dynamic in nature, should be configurable when the requirements change. Policy decisions are always made based on the current context of mobile networks. Context information refers to static and dynamic information about mobile networks. Networking Context Aware Policy Environment (NetCAPE) is a policy based mobility management architecture for such mobile networks. NetCAPE is based on the policy framework defined by the IETF and DMTF. It consists of Policy Decision Points (PDPs), Policy Enforcement Points (PEPs) and meters that work together to make decisions based on the regulatory, operator and user policies. The PDPs make decisions using the policies defined in the policy repository and the context information in context repositories. The context information is built by the meters distributed in the mobile networks. Decisions are finally executed by the PEPs that control different network elements of mobile networks. The work in this thesis consists of defining this new architecture (NetCAPE), detailing the operations and mechanisms its constituent components, implementing it and evaluating its performance against a non-NetCAPE based environment.

The paper analyzes gross upward pricing pressure indices—texttt {iGUPPI}—to assess the anti-competitive effects of mergers between vertically integrated firms where independent rivals are active in the downstream market. Such indices could be used, for example, to screen mergers between mobile network operators that compete with mobile virtual network operators in the downstream retail market. It is shown that the texttt {iGUPPI} for the downstream divisions of the merging firms corresponds to the sum of two well-known upward pricing pressure indices: the texttt {GUPPI} concept of Salop and Moresi (Georget Law J, 2009), and the texttt {vGUPPI} concept of Moresi and Salop (Antitrust Law J 79(1):185–214, 2013). However, this simple decomposition does not hold for their upstream divisions a priori. Here, additional effects that are not included in the two concepts arise. Further assumptions with respect to the price reactions of the downstream divisions to increases in the input prices are imposed so that the texttt {iGUPPI} for the upstream divisions is decomposable into an upstream market version of the texttt {GUPPI} and the texttt {vGUPPI}.

Competition in Wireless Telecommunications: The Role of MVNOs and Cable’s Entry into Wireless

This paper analyzes the spectrum trading problem in virtualized fifth generation networks to enhance the network performance with respect to the spectrum utilization. The problem is modeled as a many‐to‐many matching game with utility‐based preferences and determines the matching between mobile network operators and mobile virtual network operators. The two proposed versions of utility functions for each set aim at maximizing the satisfaction of both sets with conflicting interests and improving the overall spectrum efficiency. In the simulation evaluation, the proposed scheme is compared with three different schemes in terms of the system utility, individual, and pair matching satisfaction. We also investigate the scalability aspects, the strategy plan impact on the matching performance of our proposed scheme, and at the same time, we attempt to make appropriate assumptions closer to reality. Our proposed scheme shows much better performance than the other schemes achieving a quite high level of satisfaction for the matching result on both sets.

The throughput of a cellular network depends on a number of factors such as radio technology, limitations of device hardware (e.g., chipsets, antennae), physical layer effects (interference, fading, etc.), node density and demand, user mobility, and the infrastructure of Mobile Network Operators (MNO). Therefore, understanding and identifying the key factors of cellular network performance that affect end-users experience is a challenging task. We use a dataset collected using netradar, a platform that measures cellular network performance crowd- sourced from mobile user devices. Using this dataset we develop a methodology (a classifier using a machine learning approach) for understanding cellular network performance. We examine key characteristics of cellular networks related to throughput from the perspective of mobile user activity, MNO, smartphone models, link stability, location and time of day. We perform a network-wide correlation and statistical analysis to obtain a basic understanding of the influence of individual factors. We use a machine learning approach to identify the important features and to understand the relationship between different ones. These features are then used to build a model to classify the stability of cellular network based on the data reception characteristics of the user. We show that it is possible to classify reasons for network instability using minimal cellular network metrics with up to 90% of accuracy.

In the Internet of Things (IoT), a large number of smart mobile devices, sensors, and actuators are deployed at the edge of the network. Processing of this large volume of data is a key factor of future Internet and communication systems such as Fifth Generation (5G) cellular networks, which provide dramatic enhancement in data rates, reliability, and energy consumption as compared to existing mobile communication infrastructure. In the context of service migration (inter-Public Land Mobile Network (PLMN) roaming) between different mobile network operators, end-users experience longer service interruption due to network re-connection specifically in cross-border scenarios. In this paper, we propose Mobile-access edge computing (MEC)-based intelligent service migration architecture to improve the service continuity (SC) in multi-domain Long Term Evolution (LTE) and 5G cellular networks. We introduce the concept of virtual roaming requests processed on edge node networks and predict in advance the development of service in visiting networks. As result, we can reduce the service interruption time during service migration in a multi-domain cellular network. Moreover, this MEC-based prediction also helps to improve message communication delay and to use cellular core network resource efficiently.

In this paper, we propose hierarchical control architecture and request routing mechanisms to support N-Screen services and mobility of both sources and devices in heterogeneous networks. N-Screen services enable users to manage their active sessions among multiple devices having different capabilities, access networks and locations. To support N-Screen services in mobile operator networks, we introduce new functions that maintain the list of active devices with their sessions and network parameters for efficient N-Screen session management. The system supports the terminal and personal mobility of both the sources and devices without affecting the overall multicast connections and minimizes the service disruption. We propose a hierarchical structure for request routing and content sharing that tries to add a user to an active session locally first. The system offloads the backhaul and core networks traffic by localized session adaptation and sharing and improves the scalability issue of existing centralized mobile network architecture. We focus on the physical placement of the new nodes and mechanisms for request routing. Simulation results show that the use of hierarchical adaptation nodes in mobile operator networks can significantly reduce network latency and traffic, and improve the system throughput.

The spectrum sharing design (such as the underlay sidelink configuration in cellular networks) for vehicle-to-vehicle (V2V) and vehicle-to-network (V2N) links can improve the efficiency of spectrum utilization for vehicular networks. However, differentiated service requirements for V2V and V2N links and the greater complexity of interference management make it challenging for conventional centralized learning-based methods in the face of real-world deployment. This motivates the exploration of multi-agent deep reinforcement learning (DRL) solutions, while existing research has predominantly focused on cooperative setting that causes large communication overheads for real-world deployment. This paper proposed a decentralized resource allocation scheme for joint spectrum sub-band and V2V transmission power selection that utilizes federated learning (FL) to formulate a mixed setup to avoid frequent communication and enable easy model sharing, termed FL-MX for the design of FL and mixed DRL agent cooperation level. Different levels of cooperation among agents are also explored by implementing different state-of-the-art (SOTA) DRL-based resource allocation algorithms on the developed standard-compliant simulation platform. Through extensive simulations, it is observed that the proposed FL-MX manages to converge effectively and can match the best SOTA performance, gaining a 22.55% and 9.5% improvement in the V2V payload delivery ratio and V2N throughput compared to the random baseline. Such results make it promising for future research to investigate multi-agent DRL designs at a mixed agent cooperation level for high-performing network decision solutions with less frequent communication demands.

Due to the tremendous increase in mobile data traffic, there is a general trend toward the decentralization of mobile operator networks, at least to a certain extent. This shall be further facilitated with the virtualization of mobile network functions and the enabling of mobile cloud networking, whereby mobile networks are created on demand and in a flexible manner. Mobile network decentralization will not be efficient without rethinking mobility management schemes, particularly for users moving for a long distance and/or at a high speed (e.g., vehicles and bullet trains). To support such highly mobile users, this paper introduces: 1) a data anchor gateway (GW) relocation method based on user mobility, history information, and user activity patterns, and 2) a handover management policy that selects a target base station or evolved Node B (eNB) in a way to minimize mobility anchor GW relocation. The performance of the proposed schemes is evaluated via Markov model-based analysis and through simulations. Encouraging results are obtained, validating the design objectives of the scheme.

The optimization of channel assignment in cellular networks is a very complex optimization problem and it becomes more difficult when the network handles different classes of traffic. The objective is that channel utility be maximized so as to maximize service in a stochastic caller environment. We address the dynamic channel assignment (DCA) combined with call admission control (CAC) problem in a multimedia cellular network that handles several classes of traffic with different resource requirements. The problem is naturally formulated as a semi-Markov decision process (SMDP) problem and we use an approach based on reinforcement learning (RL) [neuro-dynamic programming (NDP)] method to solving it. We show that the policy obtained using our Q-DCA algorithm provides a good solution and is able to earn significantly higher revenues than classical solutions. A broad set of experiments illustrates the robustness of our policy that improves the quality of service (QoS) and reduces call-blocking probabilities for handoff calls in spite of variations in the traffic conditions.

Cloud computing enables the on-demand delivery of resources for a multitude of services and gives the opportunity for small agile companies to compete with large industries. In the telco world, cloud computing is currently mostly used by mobile network operators (MNO) for hosting non-critical support services and selling cloud services such as applications and data storage. MNOs are investigating the use of cloud computing to deliver key telecommunication services in the access and core networks. Without this, MNOs lose the opportunities of both combining this with over-the-top (OTT) and value-added services to their fundamental service offerings and leveraging cost-effective commodity hardware. Being able to leverage cloud computing technology effectively for the telco world is the focus of mobile cloud networking (MCN). This paper presents the key results of MCN integrated project that includes its architecture advancements, prototype implementation, and evaluation. Results show the efficiency and the simplicity that a MNO can deploy and manage the complete service lifecycle of fully cloudified, composed services that combine OTT/IT- and mobile-network-based services running on commodity hardware. The extensive performance evaluation of MCN using two key proof-of-concept scenarios that compose together many services to deliver novel converged elastic, on-demand mobile-based but innovative OTT services proves the feasibility of such fully virtualized deployments. Results show that it is beneficial to extend cloud computing to telco usage and run fully cloudified mobile-network-based systems with clear advantages and new service opportunities for MNOs and end-users.