Load Balancing With Deadline-Driven Parallel Data Transmission in Data Center Networks

High-speed routers and high-performance data centers share a common system-level architecture in which multiple processing nodes are connected by an interconnection network for high-speed communications. Load balancing is an important technique for maximizing throughput and minimizing delay of the interconnection network. In this thesis, efficient load balancing schemes are designed and analyzed for next-generation routers and data centers.
\n
\nIn high-speed router design, two preferred switch architectures are input-queued switch and load-balanced switch. In an input-queued switch, time-domain load balancing can be carried out by an iterative algorithm that schedules packets for sending in different time slots. The complexity of an iterative algorithm increases rapidly with the number of scheduling iterations. To address this problem, a single-iteration scheduling algorithm called D-LQF is designed, in which exhaustive service policy is adopted for reusing the matched input-output pairs in the previous time slots to grow the match size. 
\n
\nUnlike an input-queued switch, a load-balanced switch consists of two stages of crossbar switch fabrics, where load balancing is carried out in both time and space domains. Among various load-balanced switches, the feedback-based switch gives the best delay-throughput performance. In this thesis, the feedback-based switch is enhanced in three aspects. Firstly, we focus on reducing its switch fabric complexity. Instead of using crossbars, a dual-banyan network is proposed. The complexity of dual-banyan can be further reduced by merging the two banyans to form a Clos network, resulting in a Clos-banyan network. Secondly, we target at improving the delay performance of the feedback-based switch. A Clos-feedback switch architecture is devised where each switch module in the Clos network is a small feedback-based switch. With application-flow based load balancing, packet order is ensured and the average packet delay is reduced from O(N) to O(n), where N and n are the switch and switch module sizes, respectively. Thirdly, we extend the feedback-based switch to support multicast traffic. Based on the notion of pointer-based multicast VOQ, an efficient multicast scheduling algorithm with packet replication at the middle-stage ports only is proposed. In order to provide close-to-100% throughput for any admissible multicast traffic patterns, a three-stage implementation of feedback-based switch is also designed.
\n
\nIn designing load balancing schemes for data centers, we focus on the most popular fat-tree based data centers. Notably, packet-based load balancing is widely considered infeasible for data centers. This is because the associated packet out-of-order problem will cause unnecessary TCP fast retransmits, and as a result, severely undermine TCP performance. In this thesis, we show that if packet-based load balancing is performed properly, the packet out-of-order problem can be easily addressed by slightly increasing the number of duplicate ACKs required for triggering fast retransmit. Admittedly, in case of a real packet loss, the loss recovery time will be increased. But our simulation results show that such an increase is far less than the reduction in the network queueing delay (due to a better load-balanced network). As compared to a flow-based load balancing scheme, our packet-based scheme consistently provides significantly higher goodput and noticeably smaller delay.

Modern data center topologies often take the form of a multi-rooted tree with rich parallel paths to provide high bandwidth. However, various path diversities caused by traffic dynamics, link failures and heterogeneous switching equipments widely exist in production datacenter network. Therefore, the multi-path load balancer in data center should be robust to these diversities. Although prior fine-grained schemes such as RPS and Presto make full use of available paths, they are prone to experience packet reordering problem under asymmetric topology. The coarse-grained solutions such as ECMP and LetFlow effectively avoid packet reordering, but easily lead to under-utilization of multiple paths. To cope with these inefficiencies, we propose a load balancing mechanism called AG, which adaptively adjusts switching granularity according to the asymmetric degree of multiple paths. AG increases switching granularity to alleviate packet reordering under large degrees of topology asymmetry, while reducing switching granularity to obtain high link utilization under small degrees of topology asymmetry. AG is deployed on the switches with negligible overhead, while making no modification on end-hosts. We evaluate AG through both Mininet testbed and large-scale NS2 simulations. The experimental results show that AG reduces the average and 99th flow completion time by up to 51% and 56% over the state-of-the-art load balancing schemes, respectively.

Modern data center topologies often take the form of a multi-rooted tree with rich parallel paths to provide high bandwidth. However, various path diversities caused by traffic dynamics, link failures and heterogeneous switching equipment widely exist in the production datacenter network. Therefore, the multi-path load balancer in data center should be robust to these diversities. Although prior fine-grained schemes such as RPS and Presto make the best use of available paths, However, they are prone to experiencing packet reordering problem under the asymmetric topology. The coarse-grained solutions such as ECMP and LetFlow effectively avoid packet reordering, but easily lead to under-utilization of multiple paths. To cope with these inefficiencies, we propose a load balancing mechanism called PDLB, which adaptively adjusts flowcell granularity according to path diversity. PDLB increases flowcell granularity to alleviate packet reordering under large degrees of topology asymmetry, while reducing flowcell granularity to obtain high link utilization under small degrees of topology asymmetry. PDLB is only deployed on the sender without any modification on switch. We evaluate PDLB through large-scale NS2 simulations. The experimental results show that PDLB reduces the average flow completion time by up to ∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document}8-53% over the state-of-the-art load balancing schemes.

To meet the high performance requirement of real time and critical applications in industrial Internet of Things, modern lossless ethernet data center networks (DCNs) deployed with remote direct memory access and priority-based flow control (PFC) are dedicated to delivering low latency and high bandwidth. However, existing load balancing schemes either lack sub-round-trip-time congestion sensing or fail to accurately detect and reroute flows that cause congestion in PFC-enabled lossless DCNs. Therefore, we propose LBoDSN, an in-network load balancing for lossless DCNs using direct switch notification (DSN) for fast congestion flow detection, to address above challenges. LBoDSN tracks the evolution of ingress queue lengths at destination switches to anticipate the initiation of PFC pause, precisely identifies congested flows before PFC pause, and then sends DSNs to source switches to perform rerouting. After rerouting, the congestion notification packet associated with the previous path is selectively discarded to improve transmission performance. Experiments under realistic workloads reveal that, LBoDSN outperforms CONGA by 13%–65%, and 25%–80% in average and tail Flow Completion Times (FCTs), respectively. Compared to ConWeave, LBoDSN achieves approximately 9% improvement in both average and tail FCTs, while reducing switch queue consumption for reordering.

Hybrid grey wolf and improved particle swarm optimization with adaptive intertial weight-based multi-dimensional learning strategy for load balancing in cloud environments

Routing and wavelength assignment (RWA) is one of the key issues in the wavelength-routed optical networks. Although some RWA algorithms have been well performed to meet the need of certain networks requirement, they usually neglect the performance of the whole networks, especially the load balancing of the whole networks. This is quite likely to lead to some links bearing excessive ligthpaths and traffic load, while other links being at an idle state. In this paper, the load distribution vector ( LDV ) is introduced to describe the links load of the networks firstly. Then by means of minimizing the LDV of the networks, the load balancing of the whole networks is tried to improve. Based on this, a heuristic load balancing (HLB) strategy is presented. Moreover, a novel RWA algorithm adopting the heuristic load balancing strategy is developed, as well as two other RWA algorithms adopting other load balancing strategies. At last, the simulations of the three RWA algorithms with different load balancing strategies are done for comparison on the basis of both the regular topology and the irregular topology networks. The simulation results show that the key performance parameters such as the average variance of links, the maximum link load and the number of established lightpath are improved by means of our novel RWA algorithm with the heuristic load balancing strategy.

Load balancing and task partitioning are important components of distributed computing. The optimum performance from the distributed computing system is achieved by using effective scheduling and load balancing strategy. Researchers have well explored CPU, memory, and I/O-intensive tasks scheduling, and load balancing techniques. But one of the main obstacles of the load balancing technique leads to the ignorance of applications having a mixed nature of tasks. This is because load balancing strategies developed for one kind of job nature are not effective for the other kind of job nature. We have proposed a load balancing scheme in this paper, which is known as Mixed Task Load Balancing (MTLB) for Cluster of Workstation (CW) systems. In our proposed MTLB strategy, pre-tasks are assigned to each worker by the master to eliminate the worker's idle time. A main feature of MTLB strategy is to eradicate the inevitable selection of workers. Furthermore, the proposed MTLB strategy employs Three Resources Consideration (TRC) for load balancing (CPU, Memory, and I/O). The proposed MTLB strategy has removed the overheads of previously proposed strategies. The measured results show that MTLB strategy has a significant improvement in performance.

This paper presents the network load balancing problem, a challenging real-world task for multi-agent reinforcement learning (MARL) methods. Traditional heuristic solutions like Weighted-Cost Multi-Path (WCMP) and Local Shortest Queue (LSQ) are less flexible to the changing workload distributions and arrival rates, with a poor balance among multiple load balancers. The cooperative network load balancing task is formulated as a Dec-POMDP problem, which naturally induces the MARL methods. To bridge the reality gap for applying learning-based methods, all methods are directly trained and evaluated on an emulation system from moderate-to large-scale. Experiments on realistic testbeds show that the independent and "selfish" load balancing strategies are not necessarily the globally optimal ones, while the proposed MARL solution has a superior performance over different realistic settings. Additionally, the potential difficulties of MARL methods for network load balancing are analysed, which helps to draw the attention of the learning and network communities to such challenges.

Traditional load balancer has no flexibility to manage growing network traffic in real time and is expensive too. Software-defined networking (SDN) is a promising solution for handling today’s growing network architecture. To address issues such as network congestion and overloading, service providers use multiple replicas in the server cluster to provide the same services where network virtualization and effective load balancing is very important. This work proposes the implementation of real-time traffic management strategy that distributes the network requests among multiple servers based on a fuzzy logic. The fuzzy membership characteristic that affects the performance parameters of server load has been analyzed, and load state of virtual servers in real-time is evaluated through fuzzy logic. Considering major parameters of Web server (CPU, RAM and bandwidth utilization) as an input of fuzzy system, the clients requests are forwarded to the server having minimum load at real time. The proposed load balance algorithm written in Python is simulated on Mininet, and performance of each Web servers is compared with existing round robbin (RR) and least connections (LC) load balancing schemes. The results demonstrate that the proposed scheme has improved response time and higher throughput as compared to the other load balance solutions.KeywordsSoftware-defined networkingOpenFlowLoad balancerServer cluster

We highlight the advantages of dynamic wavelength allocation and path rerouting in hybrid optical flow-switched data center networks compared with static electronic packet-switched data center networks. Current electronic packet-switched networks perform routing based on lookup tables and shortest path routing over high speed optical connections. However, load balancing and path rerouting are not designed in coordination with the optical data plane. In this work, we design and analyze the performance of load balancing and resource allocation in hybrid flow data center architectures. In the hybrid flow architecture, the transport layer and control plane are responsible for load balancing, congestion control and reliable data delivery in wavelength division multiplexed networks. The transport layer abstracts the underlying network and data link layers from user applications. Applications are unaware of whether flows are transmitted via the electronic data plane, optical data plane, or both, where the relevant metric for applications is transaction delay. We show that the data center architecture that minimizes delay in the presence of elephant flows and dynamic loads is the hybrid flow architecture with dynamic resource allocation.

CoMap: An efficient virtual network re-mapping strategy based on coalitional matching theory

In data center, because of the large amount of user access, it usually takes the deployment of many server cluster to meet the needs of the vast numbers of users' access. How to decide a scheme to share the traffic of users to different servers becomes an important indicator of the overall quality of service in this scenario. In order to reduce costs and provide high-quality and reliable service, the load balancing technology is usually used in enterprises. At present, the industry generally uses LVS and other open source software or commercial load balancing device based on traditional network to complete this feature. However, due to the high cost of existing load balancing equipment, as well as the inherent shortcomings of traditional Internet, these solutions are lack of flexibility, and user's experience is not friendly enough. This paper proposed a load balancing scheme in data center based on SDN, that can implement an efficient dynamic scheduling according to the server status, using the feature of programmability in OpenFlow network, to balance different applications and complete the rational use of data center server cluster in a scenario that user request changes drastically. We also realized an equilibrium model for different applications. In addition, this paper also uses the Mininet network simulation tool to design the experiment scene, and completes the verification of the scheme.

In the data center environment, meeting the dissimilar requirements of transmission performance for long and short data flows is crucial for guaranteeing the data center service quality. It is well known that optimizing the network transmission performance is the key. Abundant research studies have reported that designing an effective load-balancing scheme can boost the network performance by enhancing the path utilization of multirooted data center networks. Unfortunately, the existing data center load-balancing schemes fail to provide satisfying flow-level transmission performance since they choose to ignore the constant changing path status and blindly assign paths to data flows regardless of their different performance requirements. In order to solve these issues, we propose a data center load-balancing scheme called TAM to meet the performance requirements of long and short flows simultaneously. In detail, TAM leverages the traffic-aware ECN marking mechanism to force long flows to proactively maintain the switch queue length at a low level and release bandwidth to short flows when they compete for the bottleneck link. On the contrary, TAM carries out the coarse-grained routing or rerouting for short and long flows, respectively, in real time by perceiving the path status. In this way, the short flows achieve low delay transmission while the long flows obtain high throughput. The simulation results based on NS2 tests show that TAM achieves up to 60% lower FCTs for short flows and a 40% improvement in goodput for long flows.

This paper presents large-scale analysis of traffic patterns in eighteen data centres (DCs). Data centre network (DCN) design emerges with improved architecture, as very little real information is available on existing DC traffic and characteristics. We analyzed traffic patterns of different organizations operating diverse applications. Data gathered for this analysis is over a time span of one year to observe the traffic patterns and present future predictability in network behavior. The paper presents traffic patterns and application behaviours of different applications for the observed time frames. Time frame as long-range dependence (LRD) that is maximum time span of one complete year and minimum of second time frame in small time scale (STC). The goal is to propose a traffic pattern oriented efficient bandwidth utilization and load balance by minimizing energy consumption. Our proposed framework is based on precise analysis of traffic parameters for large DCN. To improve bandwidth consumption and efficient link utilization of a network by defining a baseline threshold value that manages load balance for DC internal Traffic Engineering (TE).

Many existing load balancing mechanisms work effectively in lossy datacenter networks (DCNs), but they suffer from serious packet reordering in lossless Ethernet DCNs deployed with the hop-by-hop Priority-based Flow Control (PFC). The key reason is that the prior solutions are not able to perceive PFC triggering correctly and in a timely manner when making load balancing decisions. Once the forwarding path pauses transmission due to PFC triggering, the packets allocated on it are blocked, inevitably leading to out-of-order packets and retransmission. In this paper, we present an Reordering-robust Load Balancing (RLB) scheme with PFC prediction in lossless DCNs. At its heart, RLB leverages the derivative of ingress queue length to predict PFC triggering and proactively notifies the upstream switches to choose an appropriate rerouting path or perform packet recirculation to avoid reordering. Furthermore, under switch failure scenarios, RLB adjusts the recirculation threshold adaptively to mitigate the risk of packets over-recirculation. We have implemented RLB in the hardware programmable switch. As a building block for existing load balancing mechanisms, we have integrated RLB into Presto, LetFlow, Hermes and DRILL. The evaluation results show that the RLB-enhanced solutions deliver significant performance by avoiding packet reordering. For example, it reduces the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$99^{th}$</tex-math> </inline-formula> percentile flow completion time (FCT) by up to 72%, 67%, 58% and 54% over DRILL, Presto, LetFlow and Hermes, respectively.