RAPID: Avoiding TCP Incast Throughput Collapse in Public Clouds With Intelligent Packet Discarding

The Internet offers a large number of applications and services that we use on a daily basis. These widely used applications are hosted on large-scale, high-performance computing systems called data centers. The performance of TCP is inefficient in many-to-one communication, which is a common traffic pattern in data center networks. This many-to-one communication causes significant packet losses followed by timeouts, which consequently results in throughput collapse in data center networks; this problem is known as TCP Incast. In this paper, we present a queue length feedback-based solution to mitigate TCP Incast. The scheme has two parts: i) a novel queue length-based congestion parameter, which accurately measures congestion along the path from source to destination, and ii) a congestion control scheme that effectively uses the new congestion parameter to prevent throughput collapse due to Incast traffic patterns. Results are compared with TCP and DCTCP, the two most common transport protocols deployed in data center networks. The results show that the proposed scheme minimizes packet drops and achieves high utilization and burst tolerance.

Nowadays, TCP has been used as the de facto transport layer protocol on the Internet. However, TCP incast happens inevitably in data centers, where exist massive concurrent flows, leading to packets loss, unacceptable long transmission time, and throughput collapse. Furthermore, DCTCP is widely used to solve these problems. Unfortunately, DCTCP cannot work well under a high concurrency environment. To this end, in this paper, we introduce a two-stage congestion control algorithm (TS-TCP) for high concurrency flows. The first step is to reduce the congestion window of long-lived flows aggressively to release bandwidth for short-lived flows. The second step is to decline the sending rate by adjusting the sending interval. Through the above steps, the proposed algorithm can mitigate the TCP incast effectively. The results, by using extensive ns3 simulation, show that TS-TCP can significantly outperform the DCTCP and TCP NewReno in terms of throughput and flow completion time. Moreover, TS-TCP can support hundreds of concurrent flows while maintaining about 85\% link utilization.

Scalable applications deployed in public clouds can be built from a combination of custom software components and public cloud services. To meet performance and/or cost requirements, such applications can scale-out/in their components during run-time. When higher performance is required, new component instances can be deployed on newly allocated cloud services (e.g., virtual machines). When the instances are no longer needed, their services can be deallocated to decrease cost. However, public cloud services are usually billed over predefined time and/or usage intervals, e.g., per hour, per GB of I/O. Thus, it might not be cost efficient to scale-in public cloud applications at any moment in time, without considering their billing cycles. In this work we aid developers of scalable applications for public clouds to monitor their costs, and develop cost-aware scalability controllers. We introduce a model for capturing the pricing schemes of cloud services. Based on the model we determine and evaluate the application's costs depending on its used cloud services and their billing cycles. We further evaluate cost efficiency of cloud applications, analyzing which application component is cost efficient to deallocate and when. We evaluate our approach on a scalable platform for IoT, deployed in Flexiant1, one of the leading European public cloud providers. We show that cost-aware scalability can achieve higher application stability and performance, while reducing its operation costs.

Data center networks are characterized by high link speeds, low propagation delays, small switch buffers, and temporally clustered arrivals of many concurrent TCP flows fulfilling data transfer requests. However, the combination of these features can lead to transient buffer overflow and bursty packet losses, which in turn lead to TCP retransmission timeouts that degrade the performance of short-lived flows. This so-called TCP-incast problem can cause TCP throughput collapse. In this paper, we explore an application-level approach for solving this problem. The key idea of our solution is to coordinate the scheduling of short-lived TCP flows so that no data loss happens. We develop a mathematical model of lossless data transmission, and estimate the maximum good put achievable in data center networks. The results indicate non-monotonic good put that is highly sensitive to specific parameter configurations in the data center network. We validate our model using ns-2 network simulations, which show good correspondence with the theoretical results.

The advent of cloud computing has made access to computing infrastructure available to millions of users that face resource constraints. In the context of high performance computing (HPC), public cloud resources have emerged as a cost‐effective alternative to expensive on‐premises clusters. However, there are several challenges and limitations in adopting this approach. This paper proposes HPC@Cloud , a provider‐agnostic open‐source software toolkit that facilitates the migration, testing, and execution of HPC applications in public clouds. The toolkit takes advantage of various fault tolerance technologies to enable the use of inexpensive transient cloud infrastructure, commonly known as “spot” instances. Also, it features integration with singularity containers, allowing users to run complex applications on virtual HPC clusters in a portable and reproducible way. Finally, it provides a data‐based empirical approach to estimating cloud infrastructure costs for HPC workloads. The results obtained on two public cloud providers (AWS and Vultr) show that: (i) HPC@Cloud can efficiently build virtual HPC clusters on the cloud; (ii) the new adaptive fault tolerance strategy outperforms other existing strategies based on blocking restoration; (iii) the integration of singularity containers into HPC@Cloud improves the portability of HPC applications to public clouds with negligible performance penalty to the applications; (iv) the proposed cost prediction approach can estimate the cost of running the applications on AWS and Vultr with up to 93% accuracy on average.

The cloud computing paradigm democratized compute infrastructure access to millions of resource-strained organizations, applying economics of scale to massively reduce infrastructure costs. In the High Performance Computing (HPC) context, the benefits of using public cloud resources make it an attractive alternative to expensive on-premises clusters, however there are several challenges and limitations. In this paper, we present HPC@Cloud: a provideragnostic software framework that comprises a set of key software tools to assist in the migration, test and execution of HPC applications in public clouds. HPC@Cloud allows the HPC community to benefit from readily available public cloud resources with minimum efforts and features an empirical approach for estimating cloud infrastructure costs for HPC workloads. We also provide an experimental analysis of HPC@Cloud on two public clouds: Amazon AWS and Vultr Cloud.

Cloud computing is emerging as an alternative computing platform to bridge the gap between scientists' growing computational demands and their computing capabilities. A scientist who wants to run HPC applications can obtain massive computing resources 'in the cloud' quickly (in minutes), as opposed to days or weeks it normally takes under traditional business processes. Due to the popularity of Amazon EC2, most HPC-in-the-cloud research has been conducted using EC2 as a target platform. Previous work has not investigated how results might depend upon the cloud platform used. In this paper, we extend previous research to three public cloud computing platforms. In addition to running classical benchmarks, we also port a 'full-size' NASA climate prediction application into the cloud, and compare our results with that from dedicated HPC systems. Our results show that 1) virtualization technology, which is widely used by cloud computing, adds little performance overhead; 2) most current public clouds are not designed for running scientific applications primarily due to their poor networking capabilities. However, a cloud with moderately better network (vs. EC2) will deliver a significant performance improvement. Our observations will help to quantify the improvement of using fast networks for running HPC-in-the-cloud, and indicate a promising trend of HPC capability in future private science clouds. We also discuss techniques that will help scientists to best utilize public cloud platforms despite current deficiencies.

The problem of TCP incast in data centers attracts a lot of attention in our research community. TCP incast is a catastrophic throughput collapse that occurs when multiple senders transmitting TCP data simultaneously to a single aggregator. Based on several experiments, researchers found that TCP timeouts are the primary cause of incast problem. Particularly, timeouts due to insufficient duplicate acknowledgments is unavoidable when at least one of the last three segments is lost from the tail of a window. As a result, this type of timeouts should be avoided to improve the goodput of TCP in data center networks. A few attempts have been made to reduce timeouts, but still the problem is not completely solved especially in the case of timeouts due to insufficient duplicate acknowledgments. In this paper, we present an efficient TCP fast retransmission approach, called TCP-EFR, which is capable to reduce TCP timeouts due to lack of duplicate acknowledgments which is caused by the loss of packets from the tail of a window in data center networks. TCP-EFR makes changes in the fast retransmission and recovery algorithm of TCP by using the congestion signal mechanism of DCTCP based on instantaneous queue length. In addition, TCP-EFR controls the sending rate for avoiding the overflow of switch buffer in order to reduce the loss of packets. The results of a series of simulations in single as well as multiple bottleneck topologies using qualnet 4.5 demonstrates that TCP-EFR can significantly reduce the timeouts due to inadequate duplicate acknowledgments and noticeably improves the performance compared to DCTCP, ICTCP and TCP in terms of goodput, accuracy and stability under various network conditions.

Data center networks (DCNs) are at the forefront of a radical transformation, fueled by the rapid rise of artificial intelligence (AI), cloud-native adoption, and enterprise digitalization, which together place unprecedented demands on their design and performance. From healthcare AI and intelligent transportation to business analytics, modern DCNs are expected to support a diverse and evolving mix of bandwidth-demanding applications, each with distinct quality of service (QoS) requirements. Yet, most state-of-the-art DCN solutions rely on oversimplified DCN data traffic models, failing to address the required application-level differentiation in terms of end-to-end (E2E) latency demands. In response, we propose ALAS-DCN, a novel, to our knowledge, application-aware, low-latency, scalable DCN architecture, based on optical packet switching in the intra-rack and inter-rack domains that is both designed and rigorously evaluated using commercial QoS specifications and realistic traffic patterns of different applications coexisting within the same DCN. ALAS-DCN integrates three hierarchical optical sub-networks: intra-rack, inter-rack, and bridge, each governed by a custom wavelength division multiplexing (WDM) medium access control (MAC) protocol that dynamically prioritizes critical traffic packets, according to the specific application E2E latency demands. Unlike prior approaches, our evaluation leverages realistic workload traces generated by established traffic generators, capturing the application diversity within modern data center (DC) environments. Extensive experimental simulations across five representative DC categories, each hosting a different mix of modern applications, demonstrate that ALAS-DCN consistently achieves up to 95% throughput efficiency and sub-µs latency for high priority traffic across a variety of applications, consistently meeting their differentiated latency constraints. Scalable up to 20,000 servers, ALAS-DCN redefines the DCN as not just a transport layer but an intelligent enabler of next-generation application performance.

A survey on architectures and energy efficiency in Data Center Networks

Nowadays, the cloud computing owners lack management and monitoring tools to ensure the performance, robustness, dependability, and security. To address this limitation, this paper described our experience with a lightweight monitoring framework using some extra development work. This framework performed end-to-end measurements at virtual machine instances and software in the public cloud. It monitors quality of service parameters of the IaaS and SaaS layer without modifying the implementation of the monitored object. In addition, we discussed the manager-agent and module-centralized architecture in details. All the modules make up the entire proposed framework to improve the performance of applications in public clouds.

<p>TCP was conceived to ensure reliable node-to-node communication in moderate-bandwidth, moderate-latency, WANs. As it is now a mature Internet standard, it is the default connection-oriented protocol in networks built from commodity components, including Internet data centers. Data centers, however, rely on high-bandwidth, low-latency networks for communication. Moreover, their communication patterns, especially those generated by distributed applications such as MapReduce, often take the form of synchronous multi-node to node bursts. Under the right conditions, the network switch buffer overflow losses induced by these bursts confuse TCP's feedback mechanisms to the point that TCP throughput collapses. This collapse, termed TCP Incast, results in gross underutilization of link capacities, significantly degrading application performance.</p> <p>Conventional approaches to mitigating Incast have focused on single-path solutions, for instance, adjusting TCP's receive windows and timers, modifying the protocol itself, or adopting explicit congestion notifications. This thesis explores complementary multi-path approaches to avoiding Incast's onset. The principal idea is to use the regularity and high connectivity of typical data center networks, such as the increasingly popular fat-tree topology, to better distribute multi-node to node bursts across the available paths, thereby avoiding the switch buffer overflows that induce TCP Incast.</p> <p>The thesis's main contributions are: (1) development of new oblivious, multi-path, routing schemes for fat-tree networks, (2) derivation of relations between the schemes and Incast's onset, and (3) investigation of a novel "front-back" approach to minimizing the packet reordering introduced by multipath routing. Formal analyses are focused on relating schemes' worst-case loading of certain network resources - expressed as oblivious performance ratios (OPRs) - to Incast's onset. Potential benefits are assessed through ns-3 simulations on fat-trees under a variety of communication patterns. Results indicate that over a variety of experimental conditions, the proposed schemes reduce the incidence of TCP Incast compared to standard routing schemes.</p>

A many-to-one communication pattern is present both in Data Acquisition (DAQ) and datacenter networks. The problem arising from this pattern is widely known in the literature as incast and can be observed as TCP throughput collapse. It is a result of overloading the switch buffers, when a specific node in a network requests data from multiple sources. This paper provides two contributions. First, we confirm that there are strong analogies between the TCP behavior in DAQ and datacenter networks. Second, we evaluate different proposals from datacenter for application in DAQ to improve performance and reduce buffer requirements.

Datacenters have played an increasingly essential role as the underlying infrastructure in cloud computing. As implied by the essence of cloud computing, resources in these datacenters are shared by multiple competing entities, which can be either tenants that rent virtual machines (VMs) in a public cloud such as Amazon EC2, or applications that embrace data parallel frameworks like MapReduce in a private cloud maintained by Google. It has been generally observed that with traditional transport-layer protocols allocating link bandwidth in datacenters, network traffic from competing applications interferes with each other, resulting in a severe lack of predictability and fairness of application performance. Such a critical issue has drawn a substantial amount of recent research attention on bandwidth allocation in datacenter networks, with a number of new mechanisms proposed to efficiently and fairly share a datacenter network among competing entities. In this article, we present an extensive survey of existing bandwidth allocation mechanisms in the literature, covering the scenarios of both public and private clouds. We thoroughly investigate their underlying design principles, evaluate the trade-off involved in their design choices and summarize them in a unified design space, with the hope of conveying some meaningful insights for better designs in the future.

Data centre networks widely uses TCP for communication between the servers as it provides reliability and congestion control. However, TCP does not work well for certain type of communication patterns. One such a pattern is barrier synchronized many to one communication. During this type of data transmissions, multiple servers simultaneously transmits data to a single client and also the client cannot issues its next request until it receives all the data from current transmission. This will leads to severe throughput collapse in the networks which is called as TCP Incast. The major causes of TCP Incast are the basic TCP which is used as a transport protocol and the congestion that happens at switch. In this paper, we use TCP Lite as transport protocol instead of basic TCP and also an active queue management scheme called Random early detection Input/Output with Coupled average queue (RIO-C) is incorporated to the incast scenario. This controls the congestion that occurs due to barrier synchronized transmission. The performance analysis of RIO-C mechanism is done on the basis of metrics such as bytes received, throughput, packet loss and number of retransmissions.