Latency‐Sensitive Service Function Chains Intelligent Migration in Satellite Communication Driven by Deep Reinforcement Learning

The increasing demand in Mobile Edge Computing (MEC) networks has led to the emergence of Network Function Virtualization (NFV) technology. Through the virtualization technique, NFV can decouple each Virtual Network Function (VNF) from the hardware and deploy it flexibly in the MEC. Each service request in NFV assembles a Service Function Chain (SFC) by tying different VNFs and processes the traffic in the chained order. How to allocate resources and deploy SFC in MEC to meet Quality of Service (QoS) requirements is an important challenge for NFV. An increase in the number of VNFs with traffic passing one after the other causes high delay in SFC assembly. Also, the demanded resources may change during SFC assembly, which is clearly neglected in existing works. Therefore, high delay and resource demand uncertainty during SFC assembly affect QoS. With this motivation, we solve the SFC deployment problem by Parallelizing VNFs under Resource Demand Uncertainty (PVRDU) in MEC. We use the dependency between VNFs to transform the original SFC into a parallel SFC and then assemble several sub-SFCs based on a Deep Reinforcement Learning (DRL) approach. Meanwhile, PVRDU uses a Markov-based approximation algorithm to handle resource demand uncertainty. We perform extensive trace-driven simulations to verify the effectiveness of the proposed algorithm. The evaluation results of PVRDU are promising in different aspects compared to state-of-the-art methods. Specifically, for the considered scenario, DPC reduced SFC delay by as much as 8.7%.

Service function chaining (SFC), based on the network function virtualization (NFV) technology, is a promising candidate for realizing agile and elastic network service provisioning. This paper tackles a SFC deployment across multiple domains where only abstracted intra-domain resource information is available. For the multi-domain SFC deployment, a privacy preserving deployment mechanism called PPDM has been proposed. To conceal privacy of domains, PPDM abstracts intradomain resource information to a binary matrix, and adopts deep reinforcement learning (DRL) for flexible SFC deployment. However, because PPDM deploys all Virtual Network Functions (VNFs) included in a single SFC in a batch manner, substrate nodes need to redundantly reserve the resources required by the deployable VNFs until the construction of the SFC is completed. As a result, the deployment of the SFC may fail because of the lack of the resources on substrate nodes. In this paper, to improve acceptance ratio of SFC requests, we propose a privacy-preserving sequential deployment mechanism (PPSDM) for SFC deployment across multiple domains. The key idea behind PPSDM is to deploy VNFs in a single SFC one by one in a sequential manner to avoid redundant reservation of resources. Comprehensive simulations demonstrate that PPSDM achieves up to 22% higher acceptance ratio of SFC requests compared to the baseline approaches such as PPDM.

Low-orbit satellite communication networks have gradually become the research focus of fifth-generation (5G) beyond and sixth generation (6G) networks due to their advantages of wide coverage, large communication capacity, and low terrain influence. However, the low earth orbit mega satellite network (LEO-MSN) also has difficulty in constructing stable traffic transmission paths, network load imbalance and congestion due to the large scale of network nodes, a highly complex topology, and uneven distribution of traffic flow in time and space. In the service-based architecture proposed by 3GPP, the introduction of service function chain (SFC) constraints exacerbates these challenges. Therefore, in this paper, we propose GDRL-SFCR, an end-to-end routing decision method based on graph neural network (GNN) and deep reinforcement learning (DRL) which jointly optimize the end-to-end transmission delay and network load balancing under SFC constraints. Specifically, this method constructs the system model based on the latest NTN low-orbit satellite network end-to-end transmission architecture, taking into account the SFC constraints, transmission delays, and network node loads in the end-to-end traffic transmission, uses a GNN to extract node attributes and dynamic topology features, and uses the DRL method to design specific reward functions to train the model to learn routing policies that satisfy the SFC constraints. The simulation results demonstrate that, compared with graph theory-based methods and reinforcement learning-based methods, GDRL-SFCR can reduce the end-to-end traffic transmission delay by more than 11.3%, reduce the average network load by more than 14.1%, and increase the traffic access success rate and network capacity by more than 19.1% and two times, respectively.

Network function parallelism (NFP) has gained attention for processing packets in parallel through service functions arranged in the required service function chain. While parallel processing efficiently reduces the service function chaining (SFC) completion time, it incurs a higher network overhead (e.g., network congestion) to replicate various packets for processing. To reduce the SFC completion time while maintaining a low network overhead, we propose a deep‐reinforcement‐learning‐based NFP algorithm (DeepNFP) that provides an SFC processing policy to determine the sequential or parallel processing of every service function. In DeepNFP, deep reinforcement learning captures the network dynamics and service function conditions and iteratively finds the SFC processing policy in the network environment. Furthermore, an SFC data plane protocol based on segment routing over IPv6 configures and operates the policy in the SFC data network. Evaluation results show that DeepNFP can achieve 46% of the SFC completion time and 66% of the network overhead compared with conventional SFC and NFP, respectively.

Guest editorial IJSCN special issue on 3GPP NTN standards for future satellite communications

With the advent of Network Function Virtualization (NFV) techniques, a subset of the Internet traffic will be treated by a chain of virtual network functions (VNFs) during their journeys while the rest of the background traffic will still be carried based on traditional routing protocols. Under such a multi-service network environment, we consider the co-existence of heterogeneous traffic control mechanisms, including flexible, dynamic service function chaining (SFC) traffic control and static, dummy IP routing for the aforementioned two types of traffic that share common network resources. Depending on the traffic patterns of the background traffic which is statically routed through the traditional IP routing platform, we aim to perform dynamic service function chaining for the foreground traffic requiring VNF treatments, so that both the end-to-end SFC performance and the overall network resource utilization can be optimized. Towards this end, we propose a deep reinforcement learning based scheme to enable intelligent SFC routing decision-making in dynamic network conditions. The proposed scheme is ready to be deployed on both hybrid SDN/IP platforms and future advanced IP environments. Based on the real GEANT network topology and its one-week traffic traces, our experiments show that the proposed scheme is able to significantly improve from the traditional routing paradigm and achieve close-to-optimal performances very fast while satisfying the end-to-end SFC requirements.

The modern network is equipped with many service functions to acquire high-quality service. The emergence of network function virtualization (NFV) provides a convenient way to accomplish the network services in the form of virtual network function (VNF) and also makes the scheduling solution of VNFs flexible. The VNFs can be deployed on commodity servers as software processes. Besides, multiple VNFs are chained in a specified order as a service function chain (SFC) to serve a given flow, increasing the scheduling difficulty to minimize the average flow completion time. In this paper, we study the problem of scheduling multiple SFCs with the constraint of different resource limitations in various commodity servers. This problem is typically formulated as an Integer Linear Programming (ILP) problem, which is NP-hard. To well tackle this problem, we propose a deep reinforcement learning (DRL) approach. It involves multi-step decision making, which can be naturally transformed into a DRL problem. We design specific reward and state representations for such a multi-resource scheduling problem. We also consider how to use DRL to handle online requests of SFCs. The experiment results demonstrate that the DRL approach can significantly reduce the average completion time of a set of SFC and achieves a cost saving of 39.94% against the benchmark method.

Service Function Chains (SFC) orchestration is necessary to optimize the use of computing resources and improve the performance of the overall virtualized functions in terms of system resources cost reduction and high quality. It requires intelligent joint chaining and placement algorithm due to the evident huge amount of traffic to be delivered to end customers of the network. In this paper, a global SFC architecture and an exact approach for finding the optimal SFC components instantiation(s) (OPC) are proposed. Then, a deep reinforcement learning (DRL) approach is formulated to deal with a huge number of SFC instances. Moreover, several scenarios are considered to quantify the behavior of OPC and DRL approaches. We compare their efficiency in terms of processing cost and orchestration time. Then, different deployment flavors are implemented and assessed. To study the algorithm’s behavior and to quantify the impact of the system, novel use cases are considered. Results prove the feasibility of the exact approaches in small network scale. Still, the DRL techniques act as an heuristic approaches for chain placement in dense networks.

Service Function Chain (SFC) integrates Network Function Virtualization (NFV) for flexible deployment of Virtualized Network Function (VNF) and utilizes Software-Defined Networking (SDN) for establishing dynamic network connections, enabling customized service chains for specific applications. Security risks in NFV/SDN networks directly impact SFC, making them vulnerable to attacks. Traditional security solutions rely on passive defense mechanisms, which are insufficient against the uncertainties brought by vulnerabilities and backdoors in open networks. As an emerging network security technology, Moving Target Defense (MTD) is expected to break through traditional security protection mechanisms such as “enhanced”, “plug-in” and “passive” defense to achieve active defense against both known and unknown threats. Features of NFV/SDN networks, such as dynamic resource scheduling, microservices, and centralized control, also offer critical guidance for implementing MTD strategy. Therefore, in this paper, we propose a novel security enhancement mechanism by equipping SFC and VNFs with MTD to inherently protect SFC and VNFs during the deployment process, using a sub-pool partitioning algorithm to enhance heterogeneity across sub-pools. Additionally, to address Quality of Service (QoS) requirements, we formulate an optimization problem with three criteria: latency, cost, and defense success rate. Following that, we propose a dynamic deployment algorithm based on Deep Reinforcement Learning (DRL). Finally, to validate the effectiveness of the proposed algorithm, we conduct extensive experiment which have demonstrated that our proposed algorithm is good in increasing the security gains of both SFC and VNFs, provided that this improvement comes at the expense of modest cost and latency.

The advent of Network Function Virtualization (NFV) technology has brought flexible provisioning to Fog–Cloud Computing-based Networks (FCCNs) for enterprises to outsource their network functions to data center networks. Service Function Chaining (SFC) is a networking concept in NFV by which traffic is steered through an ordered set of Virtual Network Functions (VNFs) composing an end-to-end service. When hundreds of users outsource their network functions to FCCN, the optimal placement of VNFs in the network becomes important for assembling SFCs with the aim of resource utilization efficiency. Motivated by the scalability shortcomings of existing schemes, we propose Deep Reinforcement Learning (DRL)-based approaches by simultaneously considering parallelized SFC and reuse of VNFs to solve this problem, i.e., Asynchronous Advantage Actor–Critic (A3C). A parallelized SFC consists of several sub-SFCs, which can reduce delay and guarantee availability. Also, reuse of preliminary VNFs in SFC placement can improve computation acceleration. The proposed scheme pursues the maximization of the long-term cumulative reward for the trade-off between Quality of Service (QoS) and service cost. The results of the experiments show that the proposed scheme performs better than the state-of-the-art methods.

Network functions virtualization (NFV) solves the problem of tight coupling between network functions and hardware. While the Service Function Chaining (SFC) is the basic business model and provides customized services for network functions in NFV. However, the existing network function deployment and migration mechanisms are relatively static and rigid. For example, problems such as network load imbalance, deployment nodes or virtual network function instances failing, and virtual function elastic scaling cannot be performed. Aiming at the above problems, this paper proposes the decision-making optimization algorithm for virtual network function migration based on deep reinforcement learning. The algorithm takes network and node resources as constraints, establishes a migration decision model including state space, behavior space, and reward mechanism, and realizes flexible adaptation of virtual network functions. In the algorithm, migration decision-making based on Deep Q Network (DQN) includes offline training and online decision-making. In the offline training stage, DQN is used to train the strategy network model. The online migration stage is based on the trained strategy network model, which uses the current network state as the model input to dynamically generate the migration strategy. Finally, the network performs the migration of virtual network functions online. This algorithm optimizes the node load and solves the problem of service interruption caused by the excessive load of a single node, failure of links, and virtual function instances in the network. Finally, we build a practical prototype system and demonstrate the performance of the proposed DQN approach. According to the network status, the prototype system realizes the on-demand dynamic migration of virtual network functions. And the experimental results verify that the proposed elastic adaptation mechanism can effectively solve the migration problems, improve the quality of service, and enhance the scalability and flexibility of the network.

With the emergence of Network Function Virtualization (NFV) and Software-Defined Networks (SDN), Service Function Chaining (SFC) has evolved into a popular paradigm for carrying and fulfilling network services. However, the implementation of Mobile Edge Computing (MEC) in sixth-generation (6G) mobile networks requires efficient resource allocation mechanisms to migrate virtual network functions (VNFs). Deep learning is a promising approach to address this problem. Currently, research on VNF migration mainly focuses on how to migrate a single VNF while ignoring the VNF sharing and concurrent migration. Moreover, most existing VNF migration algorithms are complex, unscalable, and time-inefficient. This paper assumes that each placed VNF can serve multiple SFCs. We focus on selecting the best migration location for concurrently migrating VNF instances based on actual network conditions. First, we formulate the VNF migration problem as an optimization model whose goal is to minimize the end-to-end delay of all influenced SFCs while guaranteeing network load balance after migration. Next, we design a Deep Learning-based Two-Stage Algorithm (DLTSA) to solve the VNF migration problem. Finally, we combine previous experimental data to generate realistic VNF traffic patterns and evaluate the algorithm. Simulation results show that the SFC delay after migration calculated by DLTSA is close to the optimal results and much lower than the benchmarks. In addition, it effectively guarantees the load balancing of the network after migration.

Efficient instance reuse approach for service function chain placement in mobile edge computing

A cost and demand sensitive adjustment algorithm for service function chain in data center network

Network virtualization has become a promising paradigm for supporting diverse vertical services in Software Defined Networks (SDNs). Each vertical service is carried by a virtual network (VN), which normally has a chaining structure. In this way, a Service Function Chain (SFC) is composed by an ordered set of virtual network functions (VNFs) to provide tailored network services. Such new programmable flexibilities for future networks also bring new network management challenges: how to collect and analyze network measurement data, and further predict and diagnose the performance of SFCs? This is a fundamental problem for the management of SFCs, because the VNFs could be migrated in case of SFC performance degradation to avoid Service Level Agreement (SLA) violation. Despite the importance of the problem, SFC performance analysis has not attracted much research attention in the literature. In this current paper, enabled by a novel detailed network debugging technology, In-band Network Telemetry (INT), we propose a learning based framework for early SFC fault prediction and diagnosis. Based on the SFC traffic flow measurement data provided by INT, the framework firstly extracts SFC performance features. Then, Long Short-Term Memory (LSTM) networks are utilized to predict the upcoming values for these features in the next time slot. Finally, Support Vector Machine (SVM) is utilized as network fault classifier to predict possible SFC faults. We also discuss the practical utilization relevance of the proposed framework, and conduct a set of network emulations to validate the performance of the proposed framework.