Leveraging Reinforcement Learning for Traffic Engineering in Programmable Networks: A Survey

Traffic Engineering (TE) is an important network operation that routes/reroutes flows based on network topology and traffic demands to optimize network performance. Recently, new emerging applications pose challenges to TE with dynamic network conditions, where frequent routing updates are required to maintain good network performance with Software-Defined Networking (SDN). However, flow rerouting operations could lead to considerable Quality of Service (QoS) degradation and service disruption, which is often neglected by existing TE solutions. In this paper, we apply a new QoS metric named network disturbance to measure the negative impact of flow rerouting operations performed by TE. To achieve near-optimal load balancing performance and mitigate network disturbance together in dynamic network scenarios, we propose a flexible and disturbance-aware TE solution called FlexDATE that combines Reinforcement Learning (RL) and Linear Programming (LP). Specifically, FlexDATE leverages RL to intelligently identify flexible numbers of critical flows for each traffic matrix and reroutes these critical flows based on LP optimization to improve network performance with low disturbance. Empowered by a customized actor-critic architecture coupled with Graph Neural Networks (GNNs), FlexDATE can generalize well to unseen traffic scenarios and remain resilient to single link failures. Extensive simulations are conducted on five real-world network topologies to evaluate FlexDATE with real and synthetic traffic traces. The results show that FlexDATE can achieve the performance target (i.e., 90% of optimal performance) in 99% of network scenarios and effectively mitigate the average and maximum network disturbance by up to 9.1% and 38.6%, respectively, compared to state-of-the-art TE solutions.

Current implementations of Internet systems are very hard to be upgraded. The ossification of existing standards restricts the development of more advanced communication systems. New research initiatives, such as virtualization, software-defined radios, and software-defined networks, allow more flexibility for networks. However, until now, those initiatives have been developed individually. We advocate that the convergence of these overlying and complementary technologies can expand the amount of programmability on the network and support different innovative applications. Hence, this paper surveys the most recent research initiatives on programmable networks. We characterize programmable networks, where programmable devices execute specific code, and the network is separated into three planes: data, control, and management planes. We discuss the modern programmable network architectures, emphasizing their research issues, and, when possible, highlight their practical implementations. We survey the wireless and wired elements on the programmable data plane. Next, on the programmable control plane, we survey the divisor and controller elements. We conclude with final considerations, open issues and future challenges.

This paper reviews the research on Traffic Engineering (TE) based on Software-Defined Networking (SDN). Firstly, the shortcomings of traditional network architectures in managing complex traffic are introduced, and it is pointed out that SDN provides a new solution for traffic engineering through the features of separation of control and data planes, centralized control, and programmability, but at the same time, it faces many challenges. Then the three-layer architecture of SDN and its role in traffic engineering are described, and the goals, research directions and application scenarios of traffic engineering are outlined. The application of machine learning (ML) methods in SDN traffic engineering is then discussed in detail, including the advantages of deep learning (DL) in traffic prediction and classification, and the potential of reinforcement learning (RL) in routing optimization and resource scheduling. In addition, four mechanisms for optimization algorithms to merge with ML are analyzed. The integration of SDN with new architectures such as edge computing and network slicing, as well as the prospects for the application of intent-driven networking are then explored. Finally, the opportunities and challenges of SDN traffic engineering are summarized, including the advantages of flexible traffic management and cross-domain optimization, as well as the problems of network scale, security, real-time and multi-tenant management, which provide directions for future research.

Software-defined networking (SDN) is an emerging network architecture that decouples the control plane from the data plane, enabling centralized programmability and management of networks. While SDN offers benefits like flexibility, scalability, and automation, it also introduces new security vulnerabilities. This literature review analyzes the current state of research on SDN security in three key domains – Internet of Things (IoT) environments, cloud computing, and traditional enterprise networks. A systematic review methodology was followed to search, select, and review 53 relevant studies published in the past 5 years. The analysis focuses on identifying common SDN threat vectors, security solutions proposed leveraging SDN programmability and evaluating their effectiveness based on results from simulations, testbed experiments, and initial real-world implementations. Key findings of the review include lack of authentication, susceptibility to DDoS attacks, and flow rule conflicts as major security issues in SDN across domains. Dynamic traffic monitoring, access control, policy orchestration, and virtualized security functions are commonly proposed techniques to enhance SDN security. However, limitations exist in robustness testing at scale, emerging paradigms like fog computing, and quantitatively comparing SDN security with legacy networks. As SDN adoption expands, focused efforts are needed to address these research gaps through innovations in data-driven security, coordinated security policy, and emphasizing SDN controller security. This review provides valuable insights into the current state of SDN security research and informs future efforts needed in this important area

Through network programmability, we may simplify network management and bring innovation, cloud computing introduced some of its network concepts. One of the most prominent cloud models for minimizing maintenance obligations and simplifying network infrastructure administration is the SDN (Software Defined Network) architecture. SDN stands out because it provides separation of the control plane and programmability for developing network applications. As a result, SDN is expected to enable more efficient configuration, higher performance, and increased flexibility to support new network architectures. This article is aimed to demonstrates the importance of the SDN and the major role it plays in the organization and how SDNs can be profitable to many organizations that remain in the archaic or a traditional cloud environment and how SDN can restructure the cloud architecture with more security enhancement and also to investigate SDN related issues and challenges to provide insight into the obstacles that this revolutionary network paradigm will face in the future, from both a protocol and architecture standpoint. In this study, systematic literature was conducted and descriptive was used to analyze data. When it comes to SDN, the following challenges and issues stand out: All of these phrases are used to characterize the properties of a system: scalability, high availability, reliability, elasticity, security, performance, resilience, and dependability.

The exponential growth in the number of devices connected via the internet has led to the need to achieve granular programmability for increased performance, resilience, reduced latency, and jitter. Software Defined Networking (SDN) and Programming Protocol independent Packet Processing (P4) are designed to introduce programmability into the control and data plane of networks, respectively. Despite their individual potential and capabilities, the performance of combining SDN and P4 remains underexplored. This study presents a comprehensive evaluation of SDN with data plane programmability using P4 (SDN+P4) against traditional SDN with Open vSwitch (SDN+OvS), aimed at answering the hypothesis that combining SDN and P4 strengthens the control and data plane programmability and offers improved management and adaptability, which would provide a platform with faster packet processing with reduced jitter, loss, and processing overhead. Mininet was employed to emulate three distinct topologies: multi-path, grid, and transit-stub. Various traffic types were transmitted to assess performance metrics across the three topologies. Our results demonstrate that SDN+P4 outperform SDN+OvS significantly due to parallel processing, flexible parsing, and reduced overhead. The evaluation demonstrates the potential of SDN+P4 to provide a more resilient and stringent service with improved network performance for the future internet and its heterogeneity of applications.

This paper proposes a Traffic Engineering (TE) scheme for routing in a target network topology that supports OpenFlow using a metric called congestion contribution and Software-Defined Networking (SDN). According to the network traffic volume has been increasing dramatically, it is clear that the congestion, which is caused by the imbalance of the traffic flow, can be occurred. Furthermore, an Interior Gateway Protocol (IGP) that is used for an Autonomous System (AS) also can cause congestion because it uses snapshot information of network topology to make the shortest path-based routing table. Due to the reasons mentioned above, much research on TE has been conducting to enhance the network performance. However, as an example of TE, the Multi-Protocol Label Switching (MPLS) using Resource Reservation Protocol (RSVP) is not flexible in reconfiguring the labeled path, and it has resource overhead that cannot be ignored. Thus, we propose a TE scheme using SDN to overcome the limitations in the target network. In order to show the feasibility of the proposed scheme, TE application predicts the bandwidth utilization for network devices in SDN networks. It also determines the near-optimized routing path for the maximization of the average bandwidth utilization with avoiding the congestion nodes. Our scheme can configure network devices (e.g., SDN switches) along the paths of traffic flows so that the SDN network can have maximum bandwidth utilization. Through experiments, it is shown that our proposed scheme can utilize the average bandwidth of the whole topology up to 66%.

The increasing complexity and connectivity of networks lead to new and challenging vulnerabilities to be addressed for network security. As networks expand and diverse networks become interrelated, attackers leverage the broad scope of the global Internet to perpetrate large-scale attacks. Cloud environments, which are widely adopted, are fundamentally provisioned via the Internet bringing new challenges regarding security and privacy. However, network programmability and emerging technologies such as software-defined networking (SDN) and network functions virtualization (NFV) can advance techniques for intrusion detection and protection. In this chapter, we review the state of the art in scalable and collaborative intrusion detection systems (IDSs). Combining data plane programming with control level collaboration, we propose a model to simplify the detection of large-scale, distributed network attacks. This model simultaneously reduces the system overhead through direct mitigation at the network edge and enriches the detection process with corroboration of evidence from distributed sources.

Virtual Private LAN Services (VPLS) is an Ethernet based VPN (Virtual Private Network) service which provides protocol independent and high speed multipoint-to-multipoint connectivity. In this article, we discuss the possibility to use emerging networks concepts such as Software Defined Networking (SDN) and Network Function Virtualization (NFV) to improve the performance, flexibility and adaptability of VPLS networks. SDN and NFV based VPLS (SoftVPLS) architectures offer new features such as centralized control, network programmability and abstraction to improve the performance, flexibility and automation of traffic, security and network management functions for future VPLS networks.

Traditional networks are complex and hard to manage due to many reasons such as manual configuration requirements of dedicated devices, lack of flexibility and a non-dynamic approach. To overcome these limitations and to meet the challenges of modern networks a new networking paradigm Software Defined Networking (SDN) has been introduced. SDN presents a centralized and completely dynamic environment which provides flexibility and programmability in networks. It enables the network to be controlled centrally and intelligently using multiple software applications. SDN being in its infancy, brings along new challenges. Standardization of various interfaces, scalability, compatibility of contrasting or divergent networks, and vulnerability issues are few of them. This paper discusses various vulnerabilities and possible attacks on every layer of an SDN and focuses on control plane attacks. Further, it presents a comprehensive survey on numerous attacks on the brain of an SDN i.e. the control plane along with existing solutions. The study concludes that despite the challenges that are worth debating, SDN has many characteristics that make it an ideal candidate for future networks.

Programmable virtual network instantiation in IaaS cloud based on SDN

Driven by the requirement of increasing performance and flexibility, networks are being softwarized by paradigms such as software-defined networking (SDN) and network function virtualization (NFV). These solutions reduce the complexity and the specialization of hardware devices, by extracting the inherently distributed control plane of forwarding network elements such as switches and routers, to a logically centralized control plane (referred as controller in SDN). The control plane acts as a broker between the network applications (e.g. monitoring, traffic engineering) and the data plane (i.e. physical network infrastructure). For scalability and robustness, the logically centralized control plane is implemented by physically distributing different controllers throughout the network. This chapter presents different solutions to increase the reliability of both planes: data and control planes. The reliability of the data plane can be increased by considering survivable virtual network embedding solutions. This chapter proposes a survivable embedding against single and double failures at either links or nodes. Furthermore, in order to provide a programmable and resilient data plane, BPFabric has been proposed for SDN which supports high performance functions suitable for detecting attacks. On the other hand, the reliability of the control plane applied to SDN can be addressed by considering enhanced controller placement solutions providing redundancy against uncorrelated as well as targeted failures while coping with latency and capacity requirements. Furthermore, a solution to increase the security and robustness of the control channel is also addressed in this chapter.

AQROM: A quality of service aware routing optimization mechanism based on asynchronous advantage actor-critic in software-defined networks

The evolution and advancements in Information and Communication Technologies (ICT) have enabled large scale distributed computing with a huge chunk of applications for massive number of users. This has obviously generated large volumes of data, thus severely burdening the processing capacity of computers as well as the inflexible traditional networks. State-of-the-art methods for addressing datacenter level performance fixes are yet found wanting for sufficiently addressing this huge processing, storage, and network movement with proprietary protocols for this voluminous data. In this chapter the works have focused on addressing the backend server performance through effective reducer placement, intelligent compression policy, handling slower tasks and in-network performance boosting techniques through effective traffic engineering, traffic classification, topology discovery, energy minimization and load balancing in datacenter-oriented applications. Hadoop, the defacto standard in distributed big data storage and processing, has been designed to store data with its Hadoop Distributed File System (HDFS) and processing engine MapReduce large datasets reliably. However, the processing performance of Hadoop is critically dependent on the time taken to transfer data during the shuffle generated during MapReduce. Also, during concurrent execution of tasks, slower tasks need to be properly identified and efficiently handled to improve the completion time of jobs. To overcome these limitations, three contributions have been made; (i) Compression of generated map outputs at a suitable time when all the map tasks are yet to be completed to shift the load of network onto the CPU; (ii) Placing the reducer onto the nodes where the computation done is highest based on a couple of counters, one maintained at the rack level and another at node level, to minimize the run-time data copying; and (iii) Placing the slower map tasks onto the nodes where the computation done is highest and network is handled by prioritizing. Software defined networking (SDN) has been a boon for next generation networking owing to the separation of control plane from the data plane. It has the capability to address the network requirements in a timely manner by setting flows for every to and fro data movement and gathering large network statistics at the controller to make informed decisions about the network. A core issue in the network for the controller is traffic classification, which can substantially assist SDN controllers towards efficient routing and traffic engineering decisions. This chapter presents a traffic classification scheme utilizing three classifiers namely Feed-forward Neural Network (FFNN), Logistic Regression (LR), Naïve Bayes and employing Particle Swarm Optimization (PSO) for improved traffic classification with less overhead and without overlooking the key Quality of Service (QoS) criterion. Also lowering energy minimization and link utilization has been an important criterion for lowering the operating cost of the network and effectively utilizing the network. This issue has been addressed in the chapter by formulating a multi-objective problem while simultaneously addressing the QoS constraints by proposing a metaheuristic, since no polynomial solution exists and hence an evolutionary based metaheuristic (Clonal Selection) based energy optimization scheme, namely, Clonal Selection Based Energy Minimization (CSEM) has been devised. The obtained results show the efficacy of the proposed traffic classification scheme and CSEM based solution as compared with the state-of-the-art techniques. SDN has been a promising newer network paradigm but security issues and expensive capital procurement of SDN limit its full deployment hence moving to a hybrid SDN (h-SDN) deployment is only logical moving forward. The usage of both centralized and decentralized paradigms in h-SDN with intrinsic issues of interoperability poses challenges to key issues of topology gathering by the controller for proper allocation of network resources and traffic engineering for optimum network performance. State-of-the-art protocols for topology gathering, such as Link Layer Discovery Protocol (LLDP) and Broadcast Domain Discovery Protocol (BDDP) require a huge number of messages and such schemes only gather link information of SDN devices leaving out legacy switches’ (LS) links which results in sub-optimal performance. This chapter provides novel schemes which unearth topology discovery by requiring fewer messages and gathering link information of all the devices in both single and multi-controller environments (might be used when scalability issue is prevalent in h-SDN). Traffic engineering problems in h-SDN are addressed by proper placement of SDN nodes in h-SDN by utilizing the analyzing key criterion of traffic details and the degree of a node while lowering the link utilization in real-time topologies. The results of the proposed schemes for topology discovery and SDN node placement demonstrate the merits as compared with the state-of-the-art protocols.

This work focuses a wide range of modules centered ‎on latency-aware optimization strategies in order to ‎meet the increasing need for low-latency ‎communication in contemporary networks. The ‎system incorporates sophisticated congestion ‎control algorithms including LEDBAT, TCP Vegas, ‎and BBR, which regulate transmission rates more ‎efficiently than conventional loss-based techniques ‎by using delay-based metrics like round-trip time ‎‎(RTT) and queuing delay. To guarantee effective ‎path selection under latency limitations, traffic ‎engineers use multipath routing strategies as ECMP ‎and MPTCP, modified Dijkstra's algorithm with ‎latency weights, and constraint-based shortest path ‎first (CSPF). Utilizing the programmability of ‎Software-Defined Networking (SDN), the system ‎integrates metaheuristic methods including genetic ‎algorithms, ant colony optimization, and particle ‎swarm optimization along with intelligent routing ‎strategies utilizing reinforcement learning. By using ‎real-time latency feedback, these techniques allow ‎for dynamic and adaptive routing decisions. ‎OpenFlow and P4 flow rerouting features improve ‎the system's responsiveness to network conditions ‎even more. Mechanisms for monitoring and ‎feedback are essential for facilitating accurate ‎decision-making. The SDN controller's RTT ‎measurement modules continuously measure ‎connection latency, and exponential weighted ‎moving average (EWMA) methods smooth the ‎data gathered to prevent overreactions to brief ‎variations. These components work together to ‎create a strong framework for next-generation ‎network environments that optimize latency‎.