Recursively cautious congestion control

RaceCC: A rapidly converging explicit congestion control for datacenter networks

Network congestion causes severe transmission performance loss. Congestion control is the key to achieve low latency, high bandwidth and high stability. Due to the high regularity of topologies, datacenter networks contain abundant resources of equal-cost paths inherently and have huge potential for high-bandwidth transmission. Taking in-network telemetry (INT) information as the congestion signal, congestion control algorithms can monitor the network based on precise link load data. Hence the INT-based algorithm is the focus of recent high-precision congestion control research. However, existing INT-based congestion control algorithms handle the single-path transmission between the sender and the receiver, which cannot make full use of the abundant equal-cost paths resources in the network. In this paper, we make the first attempt to propose a multi-path INT-based congestion control algorithm, called MPICC. Compared with the most advanced INT-based congestion control algorithm HPCC, our method realizes multi-path coordinated transmission of data packets in datacenter networks. We conduct extensive experiments under various workloads to evaluate the performance of MPICC. The experimental results show that under the condition of ensuring high throughput of end hosts, MPICC is able to use multi-path to transmit data packets, which greatly reduces the flow completion time (FCT) of the flow in the network. Specifically, under three common datacenter workloads, Cache Follower, Web Search and Web Server, MPICC reduces the average flow completion time of the network by 20.1%, 14.4%, and 39.5%, respectively, and shortens the 97th-percentile flow completion time by 39.9%, 18.9% and 57.9%, respectively.KeywordsCongestion controlIn-network telemetry (INT)Multi-pathDatacenter networksProgrammable switch

To meet the high performance requirement of real time and critical applications in industrial Internet of Things, modern lossless ethernet data center networks (DCNs) deployed with remote direct memory access and priority-based flow control (PFC) are dedicated to delivering low latency and high bandwidth. However, existing load balancing schemes either lack sub-round-trip-time congestion sensing or fail to accurately detect and reroute flows that cause congestion in PFC-enabled lossless DCNs. Therefore, we propose LBoDSN, an in-network load balancing for lossless DCNs using direct switch notification (DSN) for fast congestion flow detection, to address above challenges. LBoDSN tracks the evolution of ingress queue lengths at destination switches to anticipate the initiation of PFC pause, precisely identifies congested flows before PFC pause, and then sends DSNs to source switches to perform rerouting. After rerouting, the congestion notification packet associated with the previous path is selectively discarded to improve transmission performance. Experiments under realistic workloads reveal that, LBoDSN outperforms CONGA by 13%–65%, and 25%–80% in average and tail Flow Completion Times (FCTs), respectively. Compared to ConWeave, LBoDSN achieves approximately 9% improvement in both average and tail FCTs, while reducing switch queue consumption for reordering.

Proactive transport has been attractive in modern high-speed and shallow buffered datacenter networks. At its heart, the link capacity is proactively allocated as credit, and then following scheduled packets are triggered by active senders according to credits, providing (near) zero loss rate and extremely low latency. Despite being promising, when waiting for credits in the first RTT, called the “pre-credit phase a substantial amount of spare bandwidth is being underutilized. To bridge this gap, current practices send a Bandwidth-Delay-Product (BDP) worth of unscheduled packets with line rate in the pre-credit phase, however, degrading throughput or tail latency. One key insight is that they consider the spare bandwidth in the pre-credit phase as a fixed value but actually changes across time and space. In this paper, we present Schef, a novel switch-based bandwidth calculation mechanism to address the pre-credit phase challenge without introducing network congestion or throughput degradation. The main idea is to calculate the spare bandwidth and apply efficient control of unscheduled packets at switches. Schef is compatible with existing proactive transports. We integrate it into a representative proposal NDP and evaluate its performance through large-scale simulations. Compared with the state-of-art, Schef reduces the average and 99th flow completion time (FCT) of short flows by up to 23% and 31%, and achieves 13% higher goodput simultaneously.

Cloud applications have reshaped the model of services and infrastructure of the Internet. Search engines, social networks, content delivery and retail and e-commerce sites belong to this group of applications. An important element in the architecture of data centers where these applications run is the communication infrastructure, commonly known as data center networks (DCNs). A critical challenge DCNs have to address is the processing of the traffic of cloud applications, which due to its properties is essentially different to the traffic of other Internet applications. In order to improve the responsiveness and throughput of applications, DCNs should be able to prioritize short flows (a few kB) over long flows (several MB). However, given the time and space variations the traffic presents, the information about flow sizes is not available in advance in order to plan the flow scheduling. There has been a wealth of solutions developed in this space, and prior work includes flow scheduling mechanisms optimizing for a specific workload but fall short when workloads are not known in advance, or comprise a collection of applications changing dynamically. In this paper, we present an adaptable mechanism called Adaptable Workload-Agnostic Flow Scheduling (AWAFS). It is an adaptable approach that can adjust in an agnostic way the scheduling configuration of DCN forwarding devices. This agnostic adjustment contributes to reduce the Flow Completion Time (FCT) of those short flows, representing around 85% of the traffic handled by cloud applications. AWAFS operates by observing the traffic and detecting statistical properties that provide a hint to adapt the scheduling parameters. Our evaluation results based on simulation show that AWAFS can reduce the average FCT of short flows between 16.9% and 45.2% when compared to the best existing agnostic non-adaptable solution, without inducing starvation on long flows. Indeed, it can provide improvements as high as 39% for long flows. Additionally, AWAFS can improve the FCT for short flows in scenarios with high heterogeneity in the traffic present in the network, with a reduction up to 5% for the average FCT and 15% for the tail FCT.

The objective of congestion control is to maximize network bandwidth utilization and minimize the average flow completion time in data center networks. Key performance indicators required to achieve this objective are high throughput and low packet latency. Existing approaches have proposed various methods to control packet delivery based on various network events or parameters such as packet loss, bottleneck bandwidth, RTT, and queue length. However, these methods often result in suboptimal packet transmission states that compromise throughput or latency. We provide a method to overcome this drawback. To this end, we introduce a congestion control method called PRTT (Predicted RTT) that leverages predicted RTT for congestion control. By using accurately predicted RTT values, PRTT dynamically adjusts the packet delivery rate to control the number of in-flight packets. This enables the transmission to approach states where the buffer holds only a few packets while fully utilizing the link bandwidth. Experimental results show that PRTT achieves higher throughput, lower latency, and shorter flow completion times than other state-of-the-art methods, particularly under bursty traffic. These results demonstrate that PRTT offers a promising solution for congestion control.

Data center technology is growing in last few years because it can accommodate variety of applications. Some data-intensive applications are deadline sensitive and require minimum flow completion time. In literature, many proposals exist for either reducing average flow completion time or the long tail of flow completion. The state of art proposal is pFabric [3] with shortest remaining size first scheduling. Data centers are generally built on layer 3 switching to reduce broadcast domain. Hosts are assigned IP address hierarchically to reduce the size of forwarding table across all data center switches. With the help of destination IP address, a sender can find the distance to the receiver. In the proposed protocol, Topology Aware pFabric (TAP), the distance information with remaining size of flow is used for flow prioritization. TAP outperforms existing protocol pFabric. The simulation results show that prioritizing long distance flows1 further reduces average flow completion time and minimizes the total number of timeout events.

Data center has become an important facility for hosting various applications. For data center networks, deadline missing rate and average flow completion time are two main metrics for the performance of applications. In this paper, we find deadline-aware methods can only reduce the percentage of flows missing deadline, while flowsize-aware and information-cumulative methods can only optimize the average flow completion time. However, traffic in data center is the mixture of various flows and focusing on the single goal is not enough. We advocate to incorporate deadline and flow duration time into flow rate control. Then we design DA&FD (Deadline-Aware and Flow Duration) based rate control mechanism and analyze its performance in theory. At last, we evaluate DA&FD under different topologies, real world traffic and load scenarios, both by simulation and in real testbed. Our results show that DA&FD performs close to D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> TCP and about 15%, 25%, 30%, 35% better than Ameon, L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DCT, Karuna, DCTCP on deadline missing rate. For average FCT, the performance of DA&FD is similar to L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DCT and compared with Ameon, D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> TCP, Karuna, DCTCP, DA&FD can reduce average FCT by 10%, 15%, 20%, 25%.

In this paper, we aim to propose a novel transmission control protocol (TCP) congestion control method from a cross-layer-based perspective and present a deep reinforcement learning (DRL)-driven method called DRL-3R (DRL for congestion control with Radio access network information and Reward Redistribution) so as to learn the TCP congestion control policy in a superior manner. In particular, we incorporate the RAN information to timely grasp the dynamics of RAN, and empower DRL to learn from the delayed RAN information feedback potentially induced by several consecutive actions. Meanwhile, we relax the implicit assumption [that the feedback to one specific action returns at a round-trip-time (RTT) after the action is applied] in previous researches, by redistributing the rewards and evaluating the merits of actions more accurately. Experiment results show that besides maintaining a reasonable fairness, DRL-3R significantly outperforms classical congestion control methods (e.g., TCP Reno, Westwood, Cubic, BBR and DRL-CC) on network utility by achieving a higher throughput while reducing delay in various network environments.

Datacenter networks employ multi-rooted topologies (e.g., Leaf-Spine, Fat-Tree) to provide large bisection bandwidth. These topologies use a large degree of multipathing, and need a data-plane load-balancing mechanism to effectively utilize their bisection bandwidth. The canonical load-balancing mechanism is equal-cost multi-path routing (ECMP), which spreads traffic uniformly across multiple paths. Motivated by ECMP's shortcomings, congestion-aware load-balancing techniques such as CONGA have been developed. These techniques have two limitations. First, because switch memory is limited, they can only maintain a small amount of congestion-tracking state at the edge switches, and do not scale to large topologies. Second, because they are implemented in custom hardware, they cannot be modified in the field.

Datacenter networks offer a large degree of multipath in order to provide large bisectional bandwidth. The end-to-end performance is determined by the load-balancing strategy which needs to be designed to effectively manage congestion. Consequently, congestion aware load-balancing strategies such as CONGA or HULA have been designed. Recently, more and more applications that are hosted on cloud servers use multipath transport protocols such as MPTCP. However, in the presence of MPTCP, existing load-balancing schemes including ECMP, HULA or CONGA may lead to suboptimal forwarding decisions where multiple MPTCP subflows of one connection are pinned on the same bottleneck link.

We propose a deadline-aware flow scheduling scheme, called Preemptive Efficient Queuing (PEQ), which takes both the deadlines and sizes of the flows into account for efficient flow scheduling of flows in a data-center network in this paper. PEQ benefits from evaluating the deadlines and flow sizes of flows to improve application throughput. Our results show that PEQ outperforms the state-of-the-art deadline-aware transport schemes in terms of the application throughput and average flow completion time, under traffic with different deadline distributions. Results also show that PEQ yields a near-optimal application throughput in cases where short flows have long deadlines and long flows have strict deadlines. In other words, PEQ attains higher performance than the counterpart schemes on traffic with broader distribution of deadlines and flow sizes.

A network rate management protocol with TCP congestion control and fairness for all

Today’s datacenter networks (DCNs) have been built upon multipath topologies where each path contains multiple links. However, flow scheduling schemes proposed to minimize flow completion times (FCT) in DCNs are based on algorithms which are optimum or close-to-optimum only over single link. Moreover, most of these scheduling schemes seek either fully centralized approaches having overhead of communicating to a central entity or fully distributed approaches requiring changes in the fabric.Motivated by these shortcomings, we present HyLine a simple scheduling design for commodity DCNs which is equipped with a joint load-balancing and flow scheduling (path-aware) design exploiting the multipath nature of DCNs. HyLine takes a hybrid approach and uses the global-awareness of centralized and agility of distributed techniques without requiring any changes in the fabric. To that end, it determines a threshold margin identifying flows for which using centralized approach is beneficial.We have shown through extensive ns2 simulations that despite HyLine’s simplicity, it significantly outperforms existing schemes and achieves lower average and 99th percentile FCTs. For instance, compared to Qjump–state-of-the-art practical scheme– and pFabric–one of the best performing flow scheduling schemes– HyLine reduces average FCT up to 68% and 31%, respectively, under a production datacenter workload.

Remote Direct Memory Access (RDMA) has been widely deployed in datacenters for its high performance. Large-scale high performance cloud services built on geographically distributed datacenters require long-range RDMA for performance requirements. However, existing RDMA solutions can hardly satisfy the stringent requirements of the emerging large-scale high-performance cloud services built on geo-distributed datacenters in terms of throughput and delay. On the one hand, lossless RDMA suffers from a deep buffer and potential suboptimal throughput for inter-datacenter traffic due to delayed response to Priority Flow Control (PFC) messages. On the other hand, lossy RDMA with selective retransmissions suffers from poor performance when multiple flows with different round-trip times (RTTs) coexist in cross-datacenter scenarios. This article proposes <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swing</small> , which expands the high-performance lossless RDMA to long-distance links through PFC-Relay. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swing</small> ensures the throughput of long-distance links while minimizing the buffer requirement for long-range RDMA. It enables long-range RDMA without making any modifications to existing in-datacenter networks. The evaluation shows that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Swing</small> can reduce the average flow completion time (FCT) by 14%-66% in a variety of traffic scenarios.