In-Network Load Balancing With Fast Congestion Flow Detection for Lossless Data Center Networks

Cloud applications have reshaped the model of services and infrastructure of the Internet. Search engines, social networks, content delivery and retail and e-commerce sites belong to this group of applications. An important element in the architecture of data centers where these applications run is the communication infrastructure, commonly known as data center networks (DCNs). A critical challenge DCNs have to address is the processing of the traffic of cloud applications, which due to its properties is essentially different to the traffic of other Internet applications. In order to improve the responsiveness and throughput of applications, DCNs should be able to prioritize short flows (a few kB) over long flows (several MB). However, given the time and space variations the traffic presents, the information about flow sizes is not available in advance in order to plan the flow scheduling. There has been a wealth of solutions developed in this space, and prior work includes flow scheduling mechanisms optimizing for a specific workload but fall short when workloads are not known in advance, or comprise a collection of applications changing dynamically. In this paper, we present an adaptable mechanism called Adaptable Workload-Agnostic Flow Scheduling (AWAFS). It is an adaptable approach that can adjust in an agnostic way the scheduling configuration of DCN forwarding devices. This agnostic adjustment contributes to reduce the Flow Completion Time (FCT) of those short flows, representing around 85% of the traffic handled by cloud applications. AWAFS operates by observing the traffic and detecting statistical properties that provide a hint to adapt the scheduling parameters. Our evaluation results based on simulation show that AWAFS can reduce the average FCT of short flows between 16.9% and 45.2% when compared to the best existing agnostic non-adaptable solution, without inducing starvation on long flows. Indeed, it can provide improvements as high as 39% for long flows. Additionally, AWAFS can improve the FCT for short flows in scenarios with high heterogeneity in the traffic present in the network, with a reduction up to 5% for the average FCT and 15% for the tail FCT.

RaceCC: A rapidly converging explicit congestion control for datacenter networks

Remote Direct Memory Access (RDMA) has been widely deployed in datacenters for its high performance. Large-scale high performance cloud services built on geographically distributed datacenters require long-range RDMA for performance requirements. However, existing RDMA solutions can hardly satisfy the stringent requirements of the emerging large-scale high-performance cloud services built on geo-distributed datacenters in terms of throughput and delay. On the one hand, lossless RDMA suffers from a deep buffer and potential suboptimal throughput for inter-datacenter traffic due to delayed response to Priority Flow Control (PFC) messages. On the other hand, lossy RDMA with selective retransmissions suffers from poor performance when multiple flows with different round-trip times (RTTs) coexist in cross-datacenter scenarios. This article proposes <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swing</small> , which expands the high-performance lossless RDMA to long-distance links through PFC-Relay. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swing</small> ensures the throughput of long-distance links while minimizing the buffer requirement for long-range RDMA. It enables long-range RDMA without making any modifications to existing in-datacenter networks. The evaluation shows that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swing</small> can reduce the average flow completion time (FCT) by 14%-66% in a variety of traffic scenarios.

Many existing load balancing mechanisms work effectively in lossy datacenter networks (DCNs), but they suffer from serious packet reordering in lossless Ethernet DCNs deployed with the hop-by-hop Priority-based Flow Control (PFC). The key reason is that the prior solutions are not able to perceive PFC triggering correctly and in a timely manner when making load balancing decisions. Once the forwarding path pauses transmission due to PFC triggering, the packets allocated on it are blocked, inevitably leading to out-of-order packets and retransmission. In this paper, we present an Reordering-robust Load Balancing (RLB) scheme with PFC prediction in lossless DCNs. At its heart, RLB leverages the derivative of ingress queue length to predict PFC triggering and proactively notifies the upstream switches to choose an appropriate rerouting path or perform packet recirculation to avoid reordering. Furthermore, under switch failure scenarios, RLB adjusts the recirculation threshold adaptively to mitigate the risk of packets over-recirculation. We have implemented RLB in the hardware programmable switch. As a building block for existing load balancing mechanisms, we have integrated RLB into Presto, LetFlow, Hermes and DRILL. The evaluation results show that the RLB-enhanced solutions deliver significant performance by avoiding packet reordering. For example, it reduces the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$99^{th}$</tex-math> </inline-formula> percentile flow completion time (FCT) by up to 72%, 67%, 58% and 54% over DRILL, Presto, LetFlow and Hermes, respectively.

Today’s datacenter networks (DCNs) have been built upon multipath topologies where each path contains multiple links. However, flow scheduling schemes proposed to minimize flow completion times (FCT) in DCNs are based on algorithms which are optimum or close-to-optimum only over single link. Moreover, most of these scheduling schemes seek either fully centralized approaches having overhead of communicating to a central entity or fully distributed approaches requiring changes in the fabric.Motivated by these shortcomings, we present HyLine a simple scheduling design for commodity DCNs which is equipped with a joint load-balancing and flow scheduling (path-aware) design exploiting the multipath nature of DCNs. HyLine takes a hybrid approach and uses the global-awareness of centralized and agility of distributed techniques without requiring any changes in the fabric. To that end, it determines a threshold margin identifying flows for which using centralized approach is beneficial.We have shown through extensive ns2 simulations that despite HyLine’s simplicity, it significantly outperforms existing schemes and achieves lower average and 99th percentile FCTs. For instance, compared to Qjump–state-of-the-art practical scheme– and pFabric–one of the best performing flow scheduling schemes– HyLine reduces average FCT up to 68% and 31%, respectively, under a production datacenter workload.

Accelerating Distributed Deep Learning draws increasing attention recently. Existing solutions like gradient compression, pipelining computation/communication, flow scheduling based on gradient disparities, neglect load balancing for Distributed Deep Learning (DDL) flows in DataCenter Network. In this paper, we review the PS traffic pattern in usual DDL training and propose that minimizing tail Flow Completion Time (FCT) of DDL flows is objective for DDL traffic Load Balancing. Besides, traffics of DDL differ from traditional DataCenter in temporal and spatial distribution. Current DCN Load Balancing become unadaptable to optimization requirements of DDL traffic, so we propose Tailor, a brand new load balancing scheme for DDL traffics training in DCN. Tailor is a solution without complex switch modification that sufficiently leverages the known parameter amount of DDL flows to discover congestion and reroutes by ECMP-like hash function to compensate bandwidth for slower flows. With large-scale simulation, we validate the effectiveness of Tailor. Tailor behaves well in both symmetric and asymmetric topology. It reduces time cost of DDL training by decreasing about 20-30% tail FCT than other load balancing mechanisms.

In the era of cloud computing, data center network (DCN) can consume a significant amount of power (e.g., 10 to 20 percent) in large-scale data centers. To reduce the power consumption of DCN, traffic consolidation has been recently proposed as an effective approach to reduce the number of DCN devices in use. However, existing consolidation approaches do not sufficiently consider the flow completion time (FCT) requirement. On one hand, missing the FCT deadlines can cause serious violation of service-level agreement, especially for delay-sensitive networking services, such as web search and E-commerce. On the other hand, keeping all the devices on to make FCTs much shorter than the desired requirements is unnecessary because 1) users may not be able to perceive the difference, and 2) such a greedy strategy can lead to unnecessarily high DCN power consumption and thus more electricity costs. In this paper, we propose FCTcon, a dynamic FCT control strategy for DCN power optimization. FCTcon is designed rigorously based on control theory to dynamically control the FCT of delay-sensitive traffic flows exactly to the requirements, such that the desired FCT performance is guaranteed while the maximum amount of DCN power savings can be achieved. Results from both hardware experiments and simulation evaluation demonstrate that, compared to the state-of-the-art DCN power optimization schemes, FCTcon can improve the DCN FCT performance, while achieving nearly the same or even more power savings. Consequently, FCTcon can result in more than 22.0 to 62.2 percent extra net profits for a data center with 50K servers. In addition, we further propose two extended designs of FCTcon to handle the coflow abstraction recently proposed for DCN, which successfully reduce the coflow deadline miss ratio by 12 to 15 percent.

Data center network (DCN) can consume a significant amount of power (e.g., 10% to 20%) in large-scale data centers. To reduce the power consumption of DCN, traffic consolidation has been recently proposed as an effective approach to reduce the number of DCN devices in use. However, existing consolidation approaches do not sufficiently consider the flow completion time (FCT) requirement. On one hand, missing the FCT deadlines can cause serious violation of service-level agreement, especially for delay-sensitive networking services, such as web search and E-commerce. On the other hand, keeping all the devices on to make FCTs much shorter than the desired requirements is unnecessary because 1) users may not be able to perceive the difference, and 2) such a greedy strategy can lead to unnecessarily high DCN power consumption and thus more electricity costs. In this paper, we propose FCTcon, a dynamic FCT control strategy for DCN power optimization. FCTcon is designed rigorously based on control theory to dynamically control the FCT of delay-sensitive traffic flows exactly to requirements, such that the desired FCT performance is guaranteed while the maximum amount of DCN power savings can be achieved. Results from both hardware experiments and simulation evaluations demonstrate that, compared to the state-of-the-art DCN power optimization schemes, FCTcon can improve the DCN FCT performance, while achieving nearly the same or even more power savings. Consequently, FCTcon can result in more than 22.0% to 62.2% extra net profits for a data center with 50K servers.

One of the major issues in Data Center Networks (DCNs) is related to elephant flows. These flows typically are primarily responsible for network congestion, causing impacts on applications that are intolerant to delays. Although there are several works in the literature that analyze this type of traffic, none of them consider the impact of elephant flow size on the throughput and on the Flow Completion Time (FCT). This article presents the EFM (Elephant Flow Manager) as a complete solution for monitoring and re-routing elephant flows in DCNs, by observing the size of these flows in the network. The tests performed show that the size of the elephant flows influences the throughput and the FCT of the applications and, therefore, deserves to be observed. The evaluation of the solution was performed comparing the joint use of EFM with ECMP (EqualCost Multi-path). In all tests, our re-routing solution proved to be better when compared to pure routing solely with ECMP. Then, this paper presents three main contributions: (1) it compares our solution with the ECMP, (2) highlights the importance of the size of the elephant flows in the re-routing of these in DCNs and (3) preliminary tests indicate that due to the decrease in FCT, it is possible to reduce the energy consumption in the data center.

Network congestion causes severe transmission performance loss. Congestion control is the key to achieve low latency, high bandwidth and high stability. Due to the high regularity of topologies, datacenter networks contain abundant resources of equal-cost paths inherently and have huge potential for high-bandwidth transmission. Taking in-network telemetry (INT) information as the congestion signal, congestion control algorithms can monitor the network based on precise link load data. Hence the INT-based algorithm is the focus of recent high-precision congestion control research. However, existing INT-based congestion control algorithms handle the single-path transmission between the sender and the receiver, which cannot make full use of the abundant equal-cost paths resources in the network. In this paper, we make the first attempt to propose a multi-path INT-based congestion control algorithm, called MPICC. Compared with the most advanced INT-based congestion control algorithm HPCC, our method realizes multi-path coordinated transmission of data packets in datacenter networks. We conduct extensive experiments under various workloads to evaluate the performance of MPICC. The experimental results show that under the condition of ensuring high throughput of end hosts, MPICC is able to use multi-path to transmit data packets, which greatly reduces the flow completion time (FCT) of the flow in the network. Specifically, under three common datacenter workloads, Cache Follower, Web Search and Web Server, MPICC reduces the average flow completion time of the network by 20.1%, 14.4%, and 39.5%, respectively, and shortens the 97th-percentile flow completion time by 39.9%, 18.9% and 57.9%, respectively.KeywordsCongestion controlIn-network telemetry (INT)Multi-pathDatacenter networksProgrammable switch

TCP's congestion control is deliberately cautious, avoiding network overloads by starting with a small initial window and then iteratively ramping up. As a result, it often takes flows several round-trip times to fully utilize the available bandwidth. In this paper we propose RC3, a technique to quickly take advantage of available capacity from the very first RTT. RC3 uses several levels of lower priority service and a modified TCP behavior to achieve near-optimal throughputs while preserving TCP-friendliness and fairness. We implement RC3 in the Linux kernel and in NS-3. In common wide-area scenarios, RC3 results in over 40% reduction in average flow completion times, with strongest improvements - more than 70% reduction in flow completion time - seen in medium to large sized (100KB - 3MB) flows.

Reducing the power consumption and maintaining the Flow Completion Time (FCT) for the Quality of Service (QoS) of applications in Data Center Networks (DCNs) are two major concerns for data center operators. However, existing works either fail in guaranteeing the QoS due to the neglect of the FCT constraints or achieve a less satisfying power efficiency. In this paper, we propose SmartFCT, which employs Software-Defined Networking (SDN) coupled with the Deep Reinforcement Learning (DRL) to improve the power efficiency of DCNs and guarantee the FCT. The DRL agent can generate a dynamic policy to consolidate traffic flows into fewer active switches in the DCN for power efficiency, and the policy also leaves different margins in different active links and switches to avoid FCT violation of unexpected short bursts of flows. Simulation results show that with similar FCT guarantee, SmartFCT can save 8% more of the power consumption compared to the state-of-the-art solutions.

The rapid development of artificial intelligence and 5G paradigm, opens up new possibilities for emerging applications in industrial Internet of Things (IIoT). However, the large amount of data, the limited resources of Internet of Things devices, and the increasing concerns of data privacy, are major obstacles to improve the quality of services in IIoT. In this article, we propose the digital twin edge networks (DITENs) by incorporating digital twin into edge networks to fill the gap between physical systems and digital spaces. We further leverage the federated learning to construct digital twin models of IoT devices based on their running data. Moreover, to mitigate the communication overhead, we propose an asynchronous model update scheme and formulate the federated learning scheme as an optimization problem. We further decompose the problem and solve the subproblems based on the deep neural network model. Numerical results show that our proposed federated learning scheme for DITEN improves the communication efficiency and reduces the transmission energy cost.

Datacenter networks are constructed of high speed networks, such as 40Gbps Ethernet, and quite severe communication quality, low latency and high throughput, is required from applications. RDMA (Remote Direct Memory Access) is now a hot topic in datacenter networking because it can operate high speed data transmission without TCP/IP protocol overhead. DCQCN (Datacenter Quantized Congestion Notification) has been proposed as congestion control for RDMA over Ethernet. In this paper, we reveal with our preliminary evaluation for DCQCN that 1) byte counter behavior causes fairness issues, 2) parameter setting of small timer for time counter and large byte counter makes time counter a dominant factor in rate increase algorithm, and 3) time counter behavior which is independent of transmission rate resolves fairness issues. And we also show that DCQCN has a technical problem of delaying transition timing to Hyper-Active Increase phase due to large byte counter setting, which causes slow convergence to the desirable transmission rate. We propose a new setting for byte counter in DCQCN so that its interval of counter increment is independent of transmission rate. This algorithm, named Adaptive Byte Counter DCQCN (ABCDCQCN), enables shorter interval of byte counter increment and can resolve both fairness issue and fast convergence of transmission rate.

Flow routing over inter-datacenter networks is a well-known problem where the network assigns a path to a newly arriving flow potentially according to the network conditions and the properties of the new flow. An essential system-wide performance metric for a routing algorithm is the flow completion times, which affect the performance of applications running across multiple datacenters. Current static and dynamic routing approaches do not take advantage of flow size information in routing, which is practical in a controlled environment such as inter-datacenter networks that are managed by the datacenter operators. In this paper, we discuss Best Worst-case Routing (BWR), which aims at optimizing the tail completion times of long-running flows over inter-datacenter networks with non-uniform link capacities. Since finding the path with the best worst-case completion time for a new flow is NP-Hard, we investigate two heuristics, BWRH and BWRHF, which use two different upper bounds on the worst-case completion times for routing. We evaluate BWRH and BWRHF against several real WAN topologies and multiple traffic patterns. Although BWRH better models the BWR problem, BWRH and BWRHF show negligible difference across various system-wide performance metrics, while BWRHF being significantly faster. Furthermore, we show that compared to other popular routing heuristics, BWRHF can reduce the mean and tail flow completion times by over 1.5× and 2×, respectively.