Enhancing Load Balancing With In-Network Recirculation to Prevent Packet Reordering in Lossless Data Centers

To meet the high performance requirement of real time and critical applications in industrial Internet of Things, modern lossless ethernet data center networks (DCNs) deployed with remote direct memory access and priority-based flow control (PFC) are dedicated to delivering low latency and high bandwidth. However, existing load balancing schemes either lack sub-round-trip-time congestion sensing or fail to accurately detect and reroute flows that cause congestion in PFC-enabled lossless DCNs. Therefore, we propose LBoDSN, an in-network load balancing for lossless DCNs using direct switch notification (DSN) for fast congestion flow detection, to address above challenges. LBoDSN tracks the evolution of ingress queue lengths at destination switches to anticipate the initiation of PFC pause, precisely identifies congested flows before PFC pause, and then sends DSNs to source switches to perform rerouting. After rerouting, the congestion notification packet associated with the previous path is selectively discarded to improve transmission performance. Experiments under realistic workloads reveal that, LBoDSN outperforms CONGA by 13%–65%, and 25%–80% in average and tail Flow Completion Times (FCTs), respectively. Compared to ConWeave, LBoDSN achieves approximately 9% improvement in both average and tail FCTs, while reducing switch queue consumption for reordering.

Modern data center topologies often take the form of a multi-rooted tree with rich parallel paths to provide high bandwidth. However, various path diversities caused by traffic dynamics, link failures and heterogeneous switching equipments widely exist in production datacenter network. Therefore, the multi-path load balancer in data center should be robust to these diversities. Although prior fine-grained schemes such as RPS and Presto make full use of available paths, they are prone to experience packet reordering problem under asymmetric topology. The coarse-grained solutions such as ECMP and LetFlow effectively avoid packet reordering, but easily lead to under-utilization of multiple paths. To cope with these inefficiencies, we propose a load balancing mechanism called AG, which adaptively adjusts switching granularity according to the asymmetric degree of multiple paths. AG increases switching granularity to alleviate packet reordering under large degrees of topology asymmetry, while reducing switching granularity to obtain high link utilization under small degrees of topology asymmetry. AG is deployed on the switches with negligible overhead, while making no modification on end-hosts. We evaluate AG through both Mininet testbed and large-scale NS2 simulations. The experimental results show that AG reduces the average and 99th flow completion time by up to 51% and 56% over the state-of-the-art load balancing schemes, respectively.

Modern data center topologies often take the form of a multi-rooted tree with rich parallel paths to provide high bandwidth. However, various path diversities caused by traffic dynamics, link failures and heterogeneous switching equipment widely exist in the production datacenter network. Therefore, the multi-path load balancer in data center should be robust to these diversities. Although prior fine-grained schemes such as RPS and Presto make the best use of available paths, However, they are prone to experiencing packet reordering problem under the asymmetric topology. The coarse-grained solutions such as ECMP and LetFlow effectively avoid packet reordering, but easily lead to under-utilization of multiple paths. To cope with these inefficiencies, we propose a load balancing mechanism called PDLB, which adaptively adjusts flowcell granularity according to path diversity. PDLB increases flowcell granularity to alleviate packet reordering under large degrees of topology asymmetry, while reducing flowcell granularity to obtain high link utilization under small degrees of topology asymmetry. PDLB is only deployed on the sender without any modification on switch. We evaluate PDLB through large-scale NS2 simulations. The experimental results show that PDLB reduces the average flow completion time by up to ∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document}8-53% over the state-of-the-art load balancing schemes.

Modern datacenters provide a wide variety of application services, which generate a mix of delay-sensitive short flows and throughput-oriented long flows, transmitting in the multi-path datacenter network. Though the existing load balancing designs successfully make full use of available parallel paths and attain high bisection network bandwidth, they reroute flows regardless of their dissimilar performance requirements. The short flows suffer from the problems of large queuing delay and packet reordering, while the long flows fail to obtain high throughput due to low link utilization and packet reordering. To address these inefficiency, we design a fine-grained load balancing scheme, namely TR (Traffic-aware Rerouting), which identifies flow types and executes flexible and traffic-aware rerouting to balance the performances of both short and long flows. Besides, to avoid packet reordering, TR leverages the reverse ACKs to estimate the switch-to-switch delay, thus excluding paths that potentially cause packet reordering. Moreover, TR is only deployed on the switch without any modification on end-hosts. The experimental results of large-scale NS2 simulations show that TR reduces the average and tail flow completion time for short flows by up to 60% and 80%, as well as provides up to 3.02x gain in throughput of long flows compared to the state-of-the-art load balancing schemes.

Accelerating Distributed Deep Learning draws increasing attention recently. Existing solutions like gradient compression, pipelining computation/communication, flow scheduling based on gradient disparities, neglect load balancing for Distributed Deep Learning (DDL) flows in DataCenter Network. In this paper, we review the PS traffic pattern in usual DDL training and propose that minimizing tail Flow Completion Time (FCT) of DDL flows is objective for DDL traffic Load Balancing. Besides, traffics of DDL differ from traditional DataCenter in temporal and spatial distribution. Current DCN Load Balancing become unadaptable to optimization requirements of DDL traffic, so we propose Tailor, a brand new load balancing scheme for DDL traffics training in DCN. Tailor is a solution without complex switch modification that sufficiently leverages the known parameter amount of DDL flows to discover congestion and reroutes by ECMP-like hash function to compensate bandwidth for slower flows. With large-scale simulation, we validate the effectiveness of Tailor. Tailor behaves well in both symmetric and asymmetric topology. It reduces time cost of DDL training by decreasing about 20-30% tail FCT than other load balancing mechanisms.

We highlight the advantages of dynamic wavelength allocation and path rerouting in hybrid optical flow-switched data center networks compared with static electronic packet-switched data center networks. Current electronic packet-switched networks perform routing based on lookup tables and shortest path routing over high speed optical connections. However, load balancing and path rerouting are not designed in coordination with the optical data plane. In this work, we design and analyze the performance of load balancing and resource allocation in hybrid flow data center architectures. In the hybrid flow architecture, the transport layer and control plane are responsible for load balancing, congestion control and reliable data delivery in wavelength division multiplexed networks. The transport layer abstracts the underlying network and data link layers from user applications. Applications are unaware of whether flows are transmitted via the electronic data plane, optical data plane, or both, where the relevant metric for applications is transaction delay. We show that the data center architecture that minimizes delay in the presence of elephant flows and dynamic loads is the hybrid flow architecture with dynamic resource allocation.

High-speed routers and high-performance data centers share a common system-level architecture in which multiple processing nodes are connected by an interconnection network for high-speed communications. Load balancing is an important technique for maximizing throughput and minimizing delay of the interconnection network. In this thesis, efficient load balancing schemes are designed and analyzed for next-generation routers and data centers.
\n
\nIn high-speed router design, two preferred switch architectures are input-queued switch and load-balanced switch. In an input-queued switch, time-domain load balancing can be carried out by an iterative algorithm that schedules packets for sending in different time slots. The complexity of an iterative algorithm increases rapidly with the number of scheduling iterations. To address this problem, a single-iteration scheduling algorithm called D-LQF is designed, in which exhaustive service policy is adopted for reusing the matched input-output pairs in the previous time slots to grow the match size. 
\n
\nUnlike an input-queued switch, a load-balanced switch consists of two stages of crossbar switch fabrics, where load balancing is carried out in both time and space domains. Among various load-balanced switches, the feedback-based switch gives the best delay-throughput performance. In this thesis, the feedback-based switch is enhanced in three aspects. Firstly, we focus on reducing its switch fabric complexity. Instead of using crossbars, a dual-banyan network is proposed. The complexity of dual-banyan can be further reduced by merging the two banyans to form a Clos network, resulting in a Clos-banyan network. Secondly, we target at improving the delay performance of the feedback-based switch. A Clos-feedback switch architecture is devised where each switch module in the Clos network is a small feedback-based switch. With application-flow based load balancing, packet order is ensured and the average packet delay is reduced from O(N) to O(n), where N and n are the switch and switch module sizes, respectively. Thirdly, we extend the feedback-based switch to support multicast traffic. Based on the notion of pointer-based multicast VOQ, an efficient multicast scheduling algorithm with packet replication at the middle-stage ports only is proposed. In order to provide close-to-100% throughput for any admissible multicast traffic patterns, a three-stage implementation of feedback-based switch is also designed.
\n
\nIn designing load balancing schemes for data centers, we focus on the most popular fat-tree based data centers. Notably, packet-based load balancing is widely considered infeasible for data centers. This is because the associated packet out-of-order problem will cause unnecessary TCP fast retransmits, and as a result, severely undermine TCP performance. In this thesis, we show that if packet-based load balancing is performed properly, the packet out-of-order problem can be easily addressed by slightly increasing the number of duplicate ACKs required for triggering fast retransmit. Admittedly, in case of a real packet loss, the loss recovery time will be increased. But our simulation results show that such an increase is far less than the reduction in the network queueing delay (due to a better load-balanced network). As compared to a flow-based load balancing scheme, our packet-based scheme consistently provides significantly higher goodput and noticeably smaller delay.

In the era of cloud computing, data center network (DCN) can consume a significant amount of power (e.g., 10 to 20 percent) in large-scale data centers. To reduce the power consumption of DCN, traffic consolidation has been recently proposed as an effective approach to reduce the number of DCN devices in use. However, existing consolidation approaches do not sufficiently consider the flow completion time (FCT) requirement. On one hand, missing the FCT deadlines can cause serious violation of service-level agreement, especially for delay-sensitive networking services, such as web search and E-commerce. On the other hand, keeping all the devices on to make FCTs much shorter than the desired requirements is unnecessary because 1) users may not be able to perceive the difference, and 2) such a greedy strategy can lead to unnecessarily high DCN power consumption and thus more electricity costs. In this paper, we propose FCTcon, a dynamic FCT control strategy for DCN power optimization. FCTcon is designed rigorously based on control theory to dynamically control the FCT of delay-sensitive traffic flows exactly to the requirements, such that the desired FCT performance is guaranteed while the maximum amount of DCN power savings can be achieved. Results from both hardware experiments and simulation evaluation demonstrate that, compared to the state-of-the-art DCN power optimization schemes, FCTcon can improve the DCN FCT performance, while achieving nearly the same or even more power savings. Consequently, FCTcon can result in more than 22.0 to 62.2 percent extra net profits for a data center with 50K servers. In addition, we further propose two extended designs of FCTcon to handle the coflow abstraction recently proposed for DCN, which successfully reduce the coflow deadline miss ratio by 12 to 15 percent.

Data center network (DCN) can consume a significant amount of power (e.g., 10% to 20%) in large-scale data centers. To reduce the power consumption of DCN, traffic consolidation has been recently proposed as an effective approach to reduce the number of DCN devices in use. However, existing consolidation approaches do not sufficiently consider the flow completion time (FCT) requirement. On one hand, missing the FCT deadlines can cause serious violation of service-level agreement, especially for delay-sensitive networking services, such as web search and E-commerce. On the other hand, keeping all the devices on to make FCTs much shorter than the desired requirements is unnecessary because 1) users may not be able to perceive the difference, and 2) such a greedy strategy can lead to unnecessarily high DCN power consumption and thus more electricity costs. In this paper, we propose FCTcon, a dynamic FCT control strategy for DCN power optimization. FCTcon is designed rigorously based on control theory to dynamically control the FCT of delay-sensitive traffic flows exactly to requirements, such that the desired FCT performance is guaranteed while the maximum amount of DCN power savings can be achieved. Results from both hardware experiments and simulation evaluations demonstrate that, compared to the state-of-the-art DCN power optimization schemes, FCTcon can improve the DCN FCT performance, while achieving nearly the same or even more power savings. Consequently, FCTcon can result in more than 22.0% to 62.2% extra net profits for a data center with 50K servers.

The stability of load balancing in the routers is a major problem for traffic flow survivability. Arrangements usually utilize an Equal Cost Multi Path (ECMP) mechanism, which basically emphasizes the load balancing network by equally splitting flows to the accessible concise paths. Empirical research conducted by previous researchers has provided new proof of techniques and methods solutions for more efficient load balancing schemes. However, a survey that examines the practical recommendation has rarely been considered. The surveys on which previous papers have focused give an experimental confirmation for load balancing in a datacenter with regards to descriptive investigation to order to suggest a scheme of best practices. The investigation of load balancing in every telecommunication layer technology is the premise for this descriptive survey. The outline arrangement of the advance level path parallelism in a datacenter and redundancy of transmission to accomplish low Flow Completion Times (FCTs) are the goals of the researcher. More research is expected to assess the effect of packet distribution against FCT. The terms usually used are load balancing, survivability, ECMP, network congestion, link criticality, and link failure protection. Load balancing or ECMP mechanism that are not focused on alternate congestion scheme of the network were exempted from the review.

One of the major issues in Data Center Networks (DCNs) is related to elephant flows. These flows typically are primarily responsible for network congestion, causing impacts on applications that are intolerant to delays. Although there are several works in the literature that analyze this type of traffic, none of them consider the impact of elephant flow size on the throughput and on the Flow Completion Time (FCT). This article presents the EFM (Elephant Flow Manager) as a complete solution for monitoring and re-routing elephant flows in DCNs, by observing the size of these flows in the network. The tests performed show that the size of the elephant flows influences the throughput and the FCT of the applications and, therefore, deserves to be observed. The evaluation of the solution was performed comparing the joint use of EFM with ECMP (EqualCost Multi-path). In all tests, our re-routing solution proved to be better when compared to pure routing solely with ECMP. Then, this paper presents three main contributions: (1) it compares our solution with the ECMP, (2) highlights the importance of the size of the elephant flows in the re-routing of these in DCNs and (3) preliminary tests indicate that due to the decrease in FCT, it is possible to reduce the energy consumption in the data center.

With the explosive growth of the Internet of Things (IoT), an increasing amount of sensor data generated by soft real-time IoT applications has been moved to data centers for storage and data analysis. Large amounts of these data are required to be processed within a given deadline to ensure application performance. Therefore, meeting the transmission deadlines of data flows for soft real-time applications has always been crucial yet challenging to current data centers. Recent progress has demonstrated that adopting parallel data transmission over multipath data center network combining with effective load balancing can achieve a high bisection network bandwidth, thus speeding up the network transfer of data flows. Nevertheless, the deadline miss ratios (DMRs) of these flows are not lowered as expected since the existing load balancing schemes are naturally agnostic to the deadline requirement. They are either unable to reroute traffic flexibly or aimlessly reroute these deadline-restrained flows, regardless of their urgent levels and path conditions. To address these inefficiencies, we propose a deadline-aware load-balancing scheme, namely, DLB, which perceives the deadline requirements and helps the urgent flows to timely switch to those faster transmission paths to complete quickly. Specifically, DLB computes the urgent level for each flow in real time to judge if the switch needs to make proactive rerouting. When a flow is nonurgent, DLB does not proactively change its transmission path, leaving more available paths to those flows with higher urgent levels. When a flow becomes extremely urgent, it immediately switches to those light-loaded paths to finish its data transmission before its deadline as far as possible. Experimental results of NS2 simulations and real testbed implementations show that DLB reduces the DMRs by up to 50% compared to the state-of-the-art data center load-balancing schemes, while only induces trivial overhead during deployment.

Cloud applications have reshaped the model of services and infrastructure of the Internet. Search engines, social networks, content delivery and retail and e-commerce sites belong to this group of applications. An important element in the architecture of data centers where these applications run is the communication infrastructure, commonly known as data center networks (DCNs). A critical challenge DCNs have to address is the processing of the traffic of cloud applications, which due to its properties is essentially different to the traffic of other Internet applications. In order to improve the responsiveness and throughput of applications, DCNs should be able to prioritize short flows (a few kB) over long flows (several MB). However, given the time and space variations the traffic presents, the information about flow sizes is not available in advance in order to plan the flow scheduling. There has been a wealth of solutions developed in this space, and prior work includes flow scheduling mechanisms optimizing for a specific workload but fall short when workloads are not known in advance, or comprise a collection of applications changing dynamically. In this paper, we present an adaptable mechanism called Adaptable Workload-Agnostic Flow Scheduling (AWAFS). It is an adaptable approach that can adjust in an agnostic way the scheduling configuration of DCN forwarding devices. This agnostic adjustment contributes to reduce the Flow Completion Time (FCT) of those short flows, representing around 85% of the traffic handled by cloud applications. AWAFS operates by observing the traffic and detecting statistical properties that provide a hint to adapt the scheduling parameters. Our evaluation results based on simulation show that AWAFS can reduce the average FCT of short flows between 16.9% and 45.2% when compared to the best existing agnostic non-adaptable solution, without inducing starvation on long flows. Indeed, it can provide improvements as high as 39% for long flows. Additionally, AWAFS can improve the FCT for short flows in scenarios with high heterogeneity in the traffic present in the network, with a reduction up to 5% for the average FCT and 15% for the tail FCT.

Routing and wavelength assignment (RWA) is one of the key issues in the wavelength-routed optical networks. Although some RWA algorithms have been well performed to meet the need of certain networks requirement, they usually neglect the performance of the whole networks, especially the load balancing of the whole networks. This is quite likely to lead to some links bearing excessive ligthpaths and traffic load, while other links being at an idle state. In this paper, the load distribution vector ( LDV ) is introduced to describe the links load of the networks firstly. Then by means of minimizing the LDV of the networks, the load balancing of the whole networks is tried to improve. Based on this, a heuristic load balancing (HLB) strategy is presented. Moreover, a novel RWA algorithm adopting the heuristic load balancing strategy is developed, as well as two other RWA algorithms adopting other load balancing strategies. At last, the simulations of the three RWA algorithms with different load balancing strategies are done for comparison on the basis of both the regular topology and the irregular topology networks. The simulation results show that the key performance parameters such as the average variance of links, the maximum link load and the number of established lightpath are improved by means of our novel RWA algorithm with the heuristic load balancing strategy.

Reducing the power consumption and maintaining the Flow Completion Time (FCT) for the Quality of Service (QoS) of applications in Data Center Networks (DCNs) are two major concerns for data center operators. However, existing works either fail in guaranteeing the QoS due to the neglect of the FCT constraints or achieve a less satisfying power efficiency. In this paper, we propose SmartFCT, which employs Software-Defined Networking (SDN) coupled with the Deep Reinforcement Learning (DRL) to improve the power efficiency of DCNs and guarantee the FCT. The DRL agent can generate a dynamic policy to consolidate traffic flows into fewer active switches in the DCN for power efficiency, and the policy also leaves different margins in different active links and switches to avoid FCT violation of unexpected short bursts of flows. Simulation results show that with similar FCT guarantee, SmartFCT can save 8% more of the power consumption compared to the state-of-the-art solutions.