On improving latency of geographically distributed key-value stores via load balancing with side information

High-speed routers and high-performance data centers share a common system-level architecture in which multiple processing nodes are connected by an interconnection network for high-speed communications. Load balancing is an important technique for maximizing throughput and minimizing delay of the interconnection network. In this thesis, efficient load balancing schemes are designed and analyzed for next-generation routers and data centers.
\n
\nIn high-speed router design, two preferred switch architectures are input-queued switch and load-balanced switch. In an input-queued switch, time-domain load balancing can be carried out by an iterative algorithm that schedules packets for sending in different time slots. The complexity of an iterative algorithm increases rapidly with the number of scheduling iterations. To address this problem, a single-iteration scheduling algorithm called D-LQF is designed, in which exhaustive service policy is adopted for reusing the matched input-output pairs in the previous time slots to grow the match size. 
\n
\nUnlike an input-queued switch, a load-balanced switch consists of two stages of crossbar switch fabrics, where load balancing is carried out in both time and space domains. Among various load-balanced switches, the feedback-based switch gives the best delay-throughput performance. In this thesis, the feedback-based switch is enhanced in three aspects. Firstly, we focus on reducing its switch fabric complexity. Instead of using crossbars, a dual-banyan network is proposed. The complexity of dual-banyan can be further reduced by merging the two banyans to form a Clos network, resulting in a Clos-banyan network. Secondly, we target at improving the delay performance of the feedback-based switch. A Clos-feedback switch architecture is devised where each switch module in the Clos network is a small feedback-based switch. With application-flow based load balancing, packet order is ensured and the average packet delay is reduced from O(N) to O(n), where N and n are the switch and switch module sizes, respectively. Thirdly, we extend the feedback-based switch to support multicast traffic. Based on the notion of pointer-based multicast VOQ, an efficient multicast scheduling algorithm with packet replication at the middle-stage ports only is proposed. In order to provide close-to-100% throughput for any admissible multicast traffic patterns, a three-stage implementation of feedback-based switch is also designed.
\n
\nIn designing load balancing schemes for data centers, we focus on the most popular fat-tree based data centers. Notably, packet-based load balancing is widely considered infeasible for data centers. This is because the associated packet out-of-order problem will cause unnecessary TCP fast retransmits, and as a result, severely undermine TCP performance. In this thesis, we show that if packet-based load balancing is performed properly, the packet out-of-order problem can be easily addressed by slightly increasing the number of duplicate ACKs required for triggering fast retransmit. Admittedly, in case of a real packet loss, the loss recovery time will be increased. But our simulation results show that such an increase is far less than the reduction in the network queueing delay (due to a better load-balanced network). As compared to a flow-based load balancing scheme, our packet-based scheme consistently provides significantly higher goodput and noticeably smaller delay.

Various key-value (KV) stores are widely employed for data management to support Internet services as they offer higher efficiency, scalability, and availability than relational database systems. The log-structured merge tree (LSM-tree) based KV stores have attracted growing attention because they can eliminate random writes and maintain acceptable read performance. Recently, as the price per unit capacity of NAND flash decreases, solid state disks (SSDs) have been extensively adopted in enterprise-scale data centers to provide high I/O bandwidth and low access latency. However, it is inefficient to naively combine LSM-tree-based KV stores with SSDs, as the high parallelism enabled within the SSD cannot be fully exploited. Current LSM-tree-based KV stores are designed without assuming SSD's multi-channel architecture.To address this inadequacy, we propose LOCS, a system equipped with a customized SSD design, which exposes its internal flash channels to applications, to work with the LSM-tree-based KV store, specifically LevelDB in this work. We extend LevelDB to explicitly leverage the multiple channels of an SSD to exploit its abundant parallelism. In addition, we optimize scheduling and dispatching polices for concurrent I/O requests to further improve the efficiency of data access. Compared with the scenario where a stock LevelDB runs on a conventional SSD, the throughput of storage system can be improved by more than 4X after applying all proposed optimization techniques.

Routing and wavelength assignment (RWA) is one of the key issues in the wavelength-routed optical networks. Although some RWA algorithms have been well performed to meet the need of certain networks requirement, they usually neglect the performance of the whole networks, especially the load balancing of the whole networks. This is quite likely to lead to some links bearing excessive ligthpaths and traffic load, while other links being at an idle state. In this paper, the load distribution vector ( LDV ) is introduced to describe the links load of the networks firstly. Then by means of minimizing the LDV of the networks, the load balancing of the whole networks is tried to improve. Based on this, a heuristic load balancing (HLB) strategy is presented. Moreover, a novel RWA algorithm adopting the heuristic load balancing strategy is developed, as well as two other RWA algorithms adopting other load balancing strategies. At last, the simulations of the three RWA algorithms with different load balancing strategies are done for comparison on the basis of both the regular topology and the irregular topology networks. The simulation results show that the key performance parameters such as the average variance of links, the maximum link load and the number of established lightpath are improved by means of our novel RWA algorithm with the heuristic load balancing strategy.

With the explosive growth of the Internet of Things (IoT), an increasing amount of sensor data generated by soft real-time IoT applications has been moved to data centers for storage and data analysis. Large amounts of these data are required to be processed within a given deadline to ensure application performance. Therefore, meeting the transmission deadlines of data flows for soft real-time applications has always been crucial yet challenging to current data centers. Recent progress has demonstrated that adopting parallel data transmission over multipath data center network combining with effective load balancing can achieve a high bisection network bandwidth, thus speeding up the network transfer of data flows. Nevertheless, the deadline miss ratios (DMRs) of these flows are not lowered as expected since the existing load balancing schemes are naturally agnostic to the deadline requirement. They are either unable to reroute traffic flexibly or aimlessly reroute these deadline-restrained flows, regardless of their urgent levels and path conditions. To address these inefficiencies, we propose a deadline-aware load-balancing scheme, namely, DLB, which perceives the deadline requirements and helps the urgent flows to timely switch to those faster transmission paths to complete quickly. Specifically, DLB computes the urgent level for each flow in real time to judge if the switch needs to make proactive rerouting. When a flow is nonurgent, DLB does not proactively change its transmission path, leaving more available paths to those flows with higher urgent levels. When a flow becomes extremely urgent, it immediately switches to those light-loaded paths to finish its data transmission before its deadline as far as possible. Experimental results of NS2 simulations and real testbed implementations show that DLB reduces the DMRs by up to 50% compared to the state-of-the-art data center load-balancing schemes, while only induces trivial overhead during deployment.

Modern distributed key-value stores are offering superior performance, incremental scalability, and fine availability for data-intensive computing and cloud-based applications. Among those distributed data stores, the designs that ensure the confidentiality of sensitive data, however, have not been fully explored yet. In this paper, we focus on designing and implementing a scalable, private, and searchable key-value store. We first design a secure data partition algorithm that distributes encrypted data evenly across a cluster of nodes. Based on this algorithm, we then implement an encrypted key-value store. To enable secure search queries for given attributes or keywords, we leverage searchable symmetric encryption to design the encrypted local indexes that consider security, efficiency, and data locality simultaneously. Performance evaluation at Microsoft Azure is conducted in terms of Put/Get throughput and latency under different workloads. The comparison with HBase shows that our prototype can function in a practical manner.

This paper presents the network load balancing problem, a challenging real-world task for multi-agent reinforcement learning (MARL) methods. Traditional heuristic solutions like Weighted-Cost Multi-Path (WCMP) and Local Shortest Queue (LSQ) are less flexible to the changing workload distributions and arrival rates, with a poor balance among multiple load balancers. The cooperative network load balancing task is formulated as a Dec-POMDP problem, which naturally induces the MARL methods. To bridge the reality gap for applying learning-based methods, all methods are directly trained and evaluated on an emulation system from moderate-to large-scale. Experiments on realistic testbeds show that the independent and "selfish" load balancing strategies are not necessarily the globally optimal ones, while the proposed MARL solution has a superior performance over different realistic settings. Additionally, the potential difficulties of MARL methods for network load balancing are analysed, which helps to draw the attention of the learning and network communities to such challenges.

Most cluster schemes select cluster head (CH) randomly without considering load balancing and security in Wireless Sensor Networks (WSNs). Our research on relevant literature shows that the existing authentication schemes do not fully consider the load balancing of CHs, while the load balancing schemes ignore the security authentication of CHs. Therefore, this paper effectively combines load balancing and security verification, and proposes a secure load balancing and authentication scheme (SLEB) based on Clustered Wireless Sensor networks. SLEB not only effectively maintains the energy balance of the whole network, but also successfully improves packet forwarding rate and prolongs the network lifetime. The simulation shows that SLEB is energy-saving, safe and sustainable compared with other similar schemes.

Key-value stores (KVS) are a critical component of current data center infrastructure. They help address the extreme demand on data centers for high bandwidth, low latency access to large amounts of data. Due to their importance, many efforts have been made to improve their performance, which includes using FPGAs to offload some functionality. These efforts have been focused on improving the performance of the key-value store itself and reducing the load on the server running the KVS. However, with more FPGAs being deployed in the data centers by many cloud service providers, some use models that were not previously practical are becoming more realistic. In this work, we explore one such use case where we attempt to cache key-value entries at the network interface of the client server. We propose an FPGA design that is capable of caching the KVS data transparently to the KVS client application. The proposed solution is able to improve the application throughput while also reducing the network traffic generated by the KVS client. Also, as the proposed solution targets client servers that are typically shared by multiple clients, we discuss the importance of, and present our vision for, an FPGA design supporting multiple tenants.

Modern data center topologies often take the form of a multi-rooted tree with rich parallel paths to provide high bandwidth. However, various path diversities caused by traffic dynamics, link failures and heterogeneous switching equipments widely exist in production datacenter network. Therefore, the multi-path load balancer in data center should be robust to these diversities. Although prior fine-grained schemes such as RPS and Presto make full use of available paths, they are prone to experience packet reordering problem under asymmetric topology. The coarse-grained solutions such as ECMP and LetFlow effectively avoid packet reordering, but easily lead to under-utilization of multiple paths. To cope with these inefficiencies, we propose a load balancing mechanism called AG, which adaptively adjusts switching granularity according to the asymmetric degree of multiple paths. AG increases switching granularity to alleviate packet reordering under large degrees of topology asymmetry, while reducing switching granularity to obtain high link utilization under small degrees of topology asymmetry. AG is deployed on the switches with negligible overhead, while making no modification on end-hosts. We evaluate AG through both Mininet testbed and large-scale NS2 simulations. The experimental results show that AG reduces the average and 99th flow completion time by up to 51% and 56% over the state-of-the-art load balancing schemes, respectively.

In data center, because of the large amount of user access, it usually takes the deployment of many server cluster to meet the needs of the vast numbers of users' access. How to decide a scheme to share the traffic of users to different servers becomes an important indicator of the overall quality of service in this scenario. In order to reduce costs and provide high-quality and reliable service, the load balancing technology is usually used in enterprises. At present, the industry generally uses LVS and other open source software or commercial load balancing device based on traditional network to complete this feature. However, due to the high cost of existing load balancing equipment, as well as the inherent shortcomings of traditional Internet, these solutions are lack of flexibility, and user's experience is not friendly enough. This paper proposed a load balancing scheme in data center based on SDN, that can implement an efficient dynamic scheduling according to the server status, using the feature of programmability in OpenFlow network, to balance different applications and complete the rational use of data center server cluster in a scenario that user request changes drastically. We also realized an equilibrium model for different applications. In addition, this paper also uses the Mininet network simulation tool to design the experiment scene, and completes the verification of the scheme.

Modern data center topologies often take the form of a multi-rooted tree with rich parallel paths to provide high bandwidth. However, various path diversities caused by traffic dynamics, link failures and heterogeneous switching equipment widely exist in the production datacenter network. Therefore, the multi-path load balancer in data center should be robust to these diversities. Although prior fine-grained schemes such as RPS and Presto make the best use of available paths, However, they are prone to experiencing packet reordering problem under the asymmetric topology. The coarse-grained solutions such as ECMP and LetFlow effectively avoid packet reordering, but easily lead to under-utilization of multiple paths. To cope with these inefficiencies, we propose a load balancing mechanism called PDLB, which adaptively adjusts flowcell granularity according to path diversity. PDLB increases flowcell granularity to alleviate packet reordering under large degrees of topology asymmetry, while reducing flowcell granularity to obtain high link utilization under small degrees of topology asymmetry. PDLB is only deployed on the sender without any modification on switch. We evaluate PDLB through large-scale NS2 simulations. The experimental results show that PDLB reduces the average flow completion time by up to ∼\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\sim$$\\end{document}8-53% over the state-of-the-art load balancing schemes.

We propose a dynamic load balancing scheme for the tele-traffic hot spot problem in cellular networks. A tele-traffic hot spot is a region of adjacent hot cells where the channel demand has exceeded a certain threshold. A hot spot is depicted as a stack of hexagonal `Rings' of cells and is classified as complete if all cells within it are hot. Otherwise it is termed incomplete. The rings containing all cold cells outside the hot spot are called `Peripheral Rings'. Our load balancing scheme migrates channels through a structured borrowing mechanism from the cold cells within the `Rings' or `Peripheral Rings' to the hot cells constituting the hot spot. A hot cell in `Ring i' can only borrow a certain fixed number of channels from adjacent cells in `Ring i+1'. We first propose a load balancing algorithm for a complete hot spot, which is then extended to the more general case of an incomplete hot spot. In the latter case, by further classifying a cell as cold safe, cold semi-safe or cold unsafe, a demand graph is constructed which describes the channel demand of each cell within the hot spot or its `Peripheral Rings' from its adjacent cells in the next outer ring. The channel borrowing algorithm works on the demand graph in a bottom up fashion, satisfying the demands of the cells in each subsequent inner ring until `Ring 0' is reached. A Markov chain model is first developed for a cell within a hot spot, the results of which are used to develop a similar model which captures the evolution of the entire hot spot region. Detailed simulation experiments are conducted to evaluate the performance of our load balancing scheme. Comparison with another well known load balancing strategy, known as CBWL, shows that under moderate and heavy tele-traffic conditions, a performance improvement as high as 12% in terms of call blockade is acheived by our load balancing scheme.

The signal intensity based wireless terminal access mechanism is the key cause of imbalanced load of WLAN. In the environment where users are densely packed, such as banks, office buildings and malls, load imbalance will severely decrease the quality and source utility rate of WLAN, thus the node load balance strategy becomes increasingly in demand to solve the problem. First, this paper studied the mainstream node load balance technologies and proposed an evaluation model which identifies and measures the level of load; next, the load level gain calculation model was designed and the prediction-based load balance and resource adjustment strategy was put forward; at last, the load balance mechanism ST-WLB was enabled and the test result shows that it can improve the performance by 10% without excessive costs.

Load balancing (LB) aims at tackling the hot-spot cell problem caused by uneven load distribution and resources shortage. Traditional LB schemes mainly focus on the load reduction of the hot-spot cell, while some partner cells might suffer from aggravating load after receiving inappropriate traffic, which triggers ripple LB and reduces the LB efficiency. To deal with this problem, a self- optimizing LB scheme in fixed relay station (RS) cellular networks is proposed. In the scheme, partner cells share the load jointly to guarantee each partner has similar load after LB. Meanwhile, the partner response mechanism considers the resources usage of partner's RSs to further reduce the ripple LB. Simulation results show that the proposed scheme can effectively reduce the number of ripple LB and reduce the call blocking rate.

Key-value stores, such as Memcached, have been used to scale web services since the beginning of the Web 2.0 era. Data center real estate is expensive, and several industry experts we have spoken to have suggested that a significant portion of their data center space is devoted to key value stores. Despite its wide-spread use, there is little in the way of hardware specialization for increasing the efficiency and density of Memcached; it is currently deployed on commodity servers that contain high-end CPUs designed to extract as much instruction-level parallelism as possible. Out-of-order CPUs, however have been shown to be inefficient when running Memcached. To address Memcached efficiency issues, we propose two architectures using 3D stacking to increase data storage efficiency. Our first 3D architecture, Mercury, consists of stacks of ARM Cortex-A7 cores with 4GB of DRAM, as well as NICs. Our second architecture, Iridium, replaces DRAM with NAND Flash to improve density. We explore, through simulation, the potential efficiency benefits of running Memcached on servers that use 3D-stacking to closely integrate low-power CPUs with NICs and memory. With Mercury we demonstrate that density may be improved by 2.9X, power efficiency by 4.9X, throughput by 10X, and throughput per GB by 3.5X over a state-of-the-art server running optimized Memcached. With Iridium we show that density may be increased by 14X, power efficiency by 2.4X, and throughput by 5.2X, while still meeting latency requirements for a majority of requests.