Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function Constraints.

The effective and reliable routing for Low Earth Orbit (LEO) satellite networks is intractable. The existing approaches cannot well handle the time-varying topology, frequent link handover, and imbalanced communication load. To tackle these issues, in this paper, we propose GRouting algorithm combining Graph Neural Networks (GNN) with Deep Reinforcement Learning (DRL) to dynamically find the optimal routing paths for LEO satellite networks. First, GNN is employed to learn the representation of satellite networks with non-Euclidean data structures. GNN is able to generalize over arbitrary satellite networks topologies, which means that it can deal with time-varying states of satellite networks. Then, based on the representation learned by GNN, DRL is applied to select the optimal routing path between two satellites, which can maximize the utilization of network resources while guaranteeing the requirement of transmission delay. Finally, extensive simulation experiments are carried out to illustrate that 1) our method has a better performance than the baseline algorithms, and 2) the GNN-based method can achieve better generalization over time-varying topologies.

ABSTRACTSatellite communication technology solves the problem that the traditional wired network infrastructure is difficult to achieve global communication coverage. However, factors such as satellite orbits introduce frequent changes to the network topology, and challenges like satellite failures and communication link interruptions are prevalent. In the face of these issues, service function chain (SFC) migration becomes a crucial method for swiftly adjusting SFCs during faults, maintaining service continuity and availability. This article proposes a latency‐sensitive SFC migration algorithm tailored to satellite networks. The algorithm first models the satellite network as a multi‐domain virtual network, capturing the constraints faced during SFC migration. Subsequently, a deep reinforcement learning algorithm integrated attention mechanism is designed to more accurately capture and understand the complex network environment and dynamic satellite network topology and derive optimal SFC migration strategies for superior performance. Finally, through experimentation and evaluation of the deep reinforcement learning‐driven latency‐sensitive service function chain intelligent migration algorithm (LS‐SFCM) in satellite communication, this study validates the effectiveness and superior performance of the algorithm in latency‐sensitive scenarios. It provides a new avenue for enhancing the service quality and efficiency of satellite communication networks.

Satellite communication networks, as an effective complement to terrestrial communication networks, enable global coverage communications. The Low-orbit satellite network has the characteristics of low transmission delay, short operation period and global coverage communication, it has become a popular research direction in satellite communication networks. With the continuous development of digital and intelligent communication in the world, satellite network services continue to expand, and higher requirements are placed on network transmission performance. The traditional single-structure low-orbit satellite network using centralized routing has been unable to meet current needs.. Therefore, with the low-orbit satellite network as the core, people have conducted in-depth research on satellite communication network routing algorithms that meet the requirements of high-quality, high-efficiency and largecapacity communication. Especially in the three hot issues of low-orbit satellite network routing algorithms, a lot of research results have emerged, including: QoS-based artificial intelligence algorithm routing algorithm; multi-layer satellite network routing algorithm; satellite network multipath routing algorithm. Through the analysis and discussion of these problems and achievements, the idea of combining the three algorithms on demand is proposed, and the challenges are analyzed.

The increasing demand in Mobile Edge Computing (MEC) networks has led to the emergence of Network Function Virtualization (NFV) technology. Through the virtualization technique, NFV can decouple each Virtual Network Function (VNF) from the hardware and deploy it flexibly in the MEC. Each service request in NFV assembles a Service Function Chain (SFC) by tying different VNFs and processes the traffic in the chained order. How to allocate resources and deploy SFC in MEC to meet Quality of Service (QoS) requirements is an important challenge for NFV. An increase in the number of VNFs with traffic passing one after the other causes high delay in SFC assembly. Also, the demanded resources may change during SFC assembly, which is clearly neglected in existing works. Therefore, high delay and resource demand uncertainty during SFC assembly affect QoS. With this motivation, we solve the SFC deployment problem by Parallelizing VNFs under Resource Demand Uncertainty (PVRDU) in MEC. We use the dependency between VNFs to transform the original SFC into a parallel SFC and then assemble several sub-SFCs based on a Deep Reinforcement Learning (DRL) approach. Meanwhile, PVRDU uses a Markov-based approximation algorithm to handle resource demand uncertainty. We perform extensive trace-driven simulations to verify the effectiveness of the proposed algorithm. The evaluation results of PVRDU are promising in different aspects compared to state-of-the-art methods. Specifically, for the considered scenario, DPC reduced SFC delay by as much as 8.7%.

Service function chaining (SFC), based on the network function virtualization (NFV) technology, is a promising candidate for realizing agile and elastic network service provisioning. This paper tackles a SFC deployment across multiple domains where only abstracted intra-domain resource information is available. For the multi-domain SFC deployment, a privacy preserving deployment mechanism called PPDM has been proposed. To conceal privacy of domains, PPDM abstracts intradomain resource information to a binary matrix, and adopts deep reinforcement learning (DRL) for flexible SFC deployment. However, because PPDM deploys all Virtual Network Functions (VNFs) included in a single SFC in a batch manner, substrate nodes need to redundantly reserve the resources required by the deployable VNFs until the construction of the SFC is completed. As a result, the deployment of the SFC may fail because of the lack of the resources on substrate nodes. In this paper, to improve acceptance ratio of SFC requests, we propose a privacy-preserving sequential deployment mechanism (PPSDM) for SFC deployment across multiple domains. The key idea behind PPSDM is to deploy VNFs in a single SFC one by one in a sequential manner to avoid redundant reservation of resources. Comprehensive simulations demonstrate that PPSDM achieves up to 22% higher acceptance ratio of SFC requests compared to the baseline approaches such as PPDM.

A new ensemble deep graph reinforcement learning network for spatio-temporal traffic volume forecasting in a freeway network

Low-Earth-Orbit (LEO) satellite networks face unique challenges in service function chain (SFC) orchestration due to their dynamic topology and resource constraints, making traditional terrestrial network solutions inadequate. This study addresses the challenge of maximizing service provider benefits through efficient SFC deployment and readjustment in LEO satellite networks. We propose PPOSFC, a novel dynamic SFC orchestration algorithm based on Proximal Policy Optimization (PPO), which incorporates future topology information into the decision-making process. The algorithm models the orchestration problem as a Markov decision process and employs a dual-objective optimization approach considering both deployment success rate and readjustment costs. Simulation results demonstrate that PPOSFC achieves a 13.07% increase in cumulative profit and improves deployment success rates by 6.78% compared to existing algorithms. The algorithm exhibits superior performance in both high and low user service request intensity, effectively balancing service quality and operational efficiency. Furthermore, our analysis reveals that incorporating predicted topology information significantly enhances orchestration performance.

Network function parallelism (NFP) has gained attention for processing packets in parallel through service functions arranged in the required service function chain. While parallel processing efficiently reduces the service function chaining (SFC) completion time, it incurs a higher network overhead (e.g., network congestion) to replicate various packets for processing. To reduce the SFC completion time while maintaining a low network overhead, we propose a deep‐reinforcement‐learning‐based NFP algorithm (DeepNFP) that provides an SFC processing policy to determine the sequential or parallel processing of every service function. In DeepNFP, deep reinforcement learning captures the network dynamics and service function conditions and iteratively finds the SFC processing policy in the network environment. Furthermore, an SFC data plane protocol based on segment routing over IPv6 configures and operates the policy in the SFC data network. Evaluation results show that DeepNFP can achieve 46% of the SFC completion time and 66% of the network overhead compared with conventional SFC and NFP, respectively.

Service Function Chains (SFC) orchestration is necessary to optimize the use of computing resources and improve the performance of the overall virtualized functions in terms of system resources cost reduction and high quality. It requires intelligent joint chaining and placement algorithm due to the evident huge amount of traffic to be delivered to end customers of the network. In this paper, a global SFC architecture and an exact approach for finding the optimal SFC components instantiation(s) (OPC) are proposed. Then, a deep reinforcement learning (DRL) approach is formulated to deal with a huge number of SFC instances. Moreover, several scenarios are considered to quantify the behavior of OPC and DRL approaches. We compare their efficiency in terms of processing cost and orchestration time. Then, different deployment flavors are implemented and assessed. To study the algorithm’s behavior and to quantify the impact of the system, novel use cases are considered. Results prove the feasibility of the exact approaches in small network scale. Still, the DRL techniques act as an heuristic approaches for chain placement in dense networks.

As an integral component of the space-air-ground integrated network (SAGIN), the low Earth orbit (LEO) satellite network has displayed immense potential in providing ubiquitous connectivity and broadband mobile communication. However, the intrinsic dynamics of LEO satellites pose unprecedented challenges in network management and service delivery. In this paper, we investigate the service function chain (SFC) orchestration in dynamic LEO satellite networks to achieve flexible and efficient service provision. Considering the service requirements and the limitations of network resources, we formulate the SFC orchestration problem as the integer nonlinear programming (INLP) problem for maximizing the service acceptance and the load fairness of satellites. Then, an efficient heuristic algorithm is proposed to solve this problem. Addressing the situation with frequent service requests, a graph attention network (GAT)-based approach with low complexity is also presented. Simulation results demonstrate that our proposed approaches outperform the benchmarks by a substantial margin in terms of load fairness and service acceptance. Besides, the proposed GAT-based approach shows its advantage in computation complexity, and exhibits robustness in unstable network scenarios with intermittent link interruptions.

The modern network is equipped with many service functions to acquire high-quality service. The emergence of network function virtualization (NFV) provides a convenient way to accomplish the network services in the form of virtual network function (VNF) and also makes the scheduling solution of VNFs flexible. The VNFs can be deployed on commodity servers as software processes. Besides, multiple VNFs are chained in a specified order as a service function chain (SFC) to serve a given flow, increasing the scheduling difficulty to minimize the average flow completion time. In this paper, we study the problem of scheduling multiple SFCs with the constraint of different resource limitations in various commodity servers. This problem is typically formulated as an Integer Linear Programming (ILP) problem, which is NP-hard. To well tackle this problem, we propose a deep reinforcement learning (DRL) approach. It involves multi-step decision making, which can be naturally transformed into a DRL problem. We design specific reward and state representations for such a multi-resource scheduling problem. We also consider how to use DRL to handle online requests of SFCs. The experiment results demonstrate that the DRL approach can significantly reduce the average completion time of a set of SFC and achieves a cost saving of 39.94% against the benchmark method.

Service Function Chain (SFC) integrates Network Function Virtualization (NFV) for flexible deployment of Virtualized Network Function (VNF) and utilizes Software-Defined Networking (SDN) for establishing dynamic network connections, enabling customized service chains for specific applications. Security risks in NFV/SDN networks directly impact SFC, making them vulnerable to attacks. Traditional security solutions rely on passive defense mechanisms, which are insufficient against the uncertainties brought by vulnerabilities and backdoors in open networks. As an emerging network security technology, Moving Target Defense (MTD) is expected to break through traditional security protection mechanisms such as “enhanced”, “plug-in” and “passive” defense to achieve active defense against both known and unknown threats. Features of NFV/SDN networks, such as dynamic resource scheduling, microservices, and centralized control, also offer critical guidance for implementing MTD strategy. Therefore, in this paper, we propose a novel security enhancement mechanism by equipping SFC and VNFs with MTD to inherently protect SFC and VNFs during the deployment process, using a sub-pool partitioning algorithm to enhance heterogeneity across sub-pools. Additionally, to address Quality of Service (QoS) requirements, we formulate an optimization problem with three criteria: latency, cost, and defense success rate. Following that, we propose a dynamic deployment algorithm based on Deep Reinforcement Learning (DRL). Finally, to validate the effectiveness of the proposed algorithm, we conduct extensive experiment which have demonstrated that our proposed algorithm is good in increasing the security gains of both SFC and VNFs, provided that this improvement comes at the expense of modest cost and latency.

Stringent delay requirements of many mobile applications have led to the development of mobile edge clouds, to offer low latency network services at the network edges. Most conventional network services are implemented via hardware-based network functions, including firewalls and load balancers, to guarantee service security and performance. However, implementing hardware-based network functions usually incurs both a high capital expenditure (CAPEX) and operating expenditure (OPEX). Network Function Virtualization (NFV) exhibits a potential to reduce CAPEX and OPEX significantly, by deploying software-based network functions in virtual machines (VMs) on edge-clouds. We consider a fundamental problem of NFV-enabled multicasting in a mobile edge cloud, where each multicast request has both service function chain and end-to-end delay requirements. Specifically, each multicast request requires chaining of a sequence of network functions (referred to as a service function chain) from a source to a set of destinations within specified end-to-end delay requirements. We devise an approximation algorithm with a provable approximation ratio for a single multicast request admission if its delay requirement is negligible; otherwise, we propose an efficient heuristic. Furthermore, we also consider admissions of a given set of the delay-aware NFV-enabled multicast requests, for which we devise an efficient heuristic such that the system throughput is maximized, while the implementation cost of admitted requests is minimized. We finally evaluate the performance of the proposed algorithms in a real test-bed, and experimental results show that our algorithms outperform other similar approaches reported in literature.

Software-defined networking (SDN) and the network function virtualization (NFV) led to great developments in software based control technology by decreasing expenditures. Service function chaining (SFC) is an important technology to find efficient paths in network servers to process all of the requested virtualized network functions (VNF). However, SFC is challenging since it has to maintain high Quality of Service (QoS) even for complicated situations. Although some works have been conducted for such tasks with high-level intelligent models like deep neural networks (DNNs), those approaches are not efficient in utilizing the topology information of networks and cannot be applied to networks with dynamically changing topology since their models assume that the topology is fixed. In this paper, we propose a new neural network architecture for SFC, which is based on graph neural network (GNN) considering the graph-structured properties of network topology. The proposed SFC model consists of an encoder and a decoder, where the encoder finds the representation of the network topology, and then the decoder estimates probabilities of neighborhood nodes and their probabilities to process a VNF. In the experiments, our proposed architecture outperformed previous performances of DNN based baseline model. Moreover, the GNN based model can be applied to a new network topology without re-designing and re-training.

Software Defined Satellite Networks (SDSN) are proposed to solve the problems in traditional satellite networks, such as time-consuming configuration and inflexible traffic scheduling. The emerging application of small satellite and research of SDSN make it possible for satellite networks to provide flexible network services. Service Function Chain (SFC) can satisfy this need. In this paper, we are motivated to investigate applying SFC in the small satellite-based SDSN for service delivery. We introduce the structure of the multi-layer constellation-based SDSN. Then, we describe two deployment patterns of SFC in SDSN, the Multi-Domain (MD) pattern and the Satellite Formation (SF) pattern. We propose two algorithms, SFP-MD, and SFP-SF, to calculate the Service Function Path (SFP). We implement the algorithms and conduct contrast experiments in our prototype. Finally, we summarize the applicable conditions of two deployment patterns according to the experimental results in terms of hops, delay, and packet loss rate.