• Home
  • Search
  • Work stealing for interactive services to meet target latency
  • Cite Icon40
  • https://doi.org/10.1145/2851141.2851151Copy DOI Icon

Work stealing for interactive services to meet target latency

  • Feb 27, 2016
  • Jing Li +6 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Interactive web services increasingly drive critical business workloads such as search, advertising, games, shopping, and finance. Whereas optimizing parallel programs and distributed server systems have historically focused on average latency and throughput, the primary metric for interactive applications is instead consistent responsiveness, i.e., minimizing the number of requests that miss a target latency. This paper is the first to show how to generalize work-stealing, which is traditionally used to minimize the makespan of a single parallel job, to optimize for a target latency in interactive services with multiple parallel requests.We design a new adaptive work stealing policy, called tail-control, that reduces the number of requests that miss a target latency. It uses instantaneous request progress, system load, and a target latency to choose when to parallelize requests with stealing, when to admit new requests, and when to limit parallelism of large requests. We implement this approach in the Intel Thread Building Block (TBB) library and evaluate it on real-world workloads and synthetic workloads. The tail-control policy substantially reduces the number of requests exceeding the desired target latency and delivers up to 58% relative improvement over various baseline policies. This generalization of work stealing for multiple requests effectively optimizes the number of requests that complete within a target latency, a key metric for interactive services.

Similar Papers
  • Research Article
  • Citations27

A runtime system for interactive Web services

  • May 01, 1999
  • Computer Networks
  • Claus Brabrand +3
  • Book Chapter
  • Citations34

WS-Engineer: A Model-Based Approach to Engineering Web Service Compositions and Choreography

  • Jan 01, 2007
  • Howard Foster +3
  • Research Article
  • Citations3

A Contract-aware and Cost-effective LSM Store for Cloud Storage with Low Latency Spikes

  • Apr 04, 2024
  • ACM Transactions on Storage
  • Yuanhui Zhou +8
  • Conference Article
  • Citations35

A type system for dynamic Web documents

  • Jan 05, 2000
  • Anders Sandholm +1
  • Conference Article
  • Citations3

Synthesis of Reconfigurable Machine Tools With Screw Theory

  • Sep 10, 2000
  • Yong-Mo Moon +1
  • Research Article
  • Citations19

A low overhead, fault tolerant and congestion aware routing algorithm for 3D mesh-based Network-on-Chips

  • Oct 05, 2014
  • Microprocessors and Microsystems
  • Hoda Naghibi Jouybari +1
  • Book Chapter

Towards a Collaborative, Interactive Web Services Composition Approach Based on an Intentional Group Recommender System

  • Jan 01, 2018
  • Meriem Kasmi +2
  • Conference Article
  • Citations45

Cluster load balancing for fine-grain network services

  • Jan 01, 2002
  • Kai Shen +2
  • Research Article
  • Citations8

A 0.9-V microcontroller for portable applications

  • Jul 01, 1997
  • IEEE Journal of Solid-State Circuits
  • Andreas Wild +25
  • Research Article
  • Citations62

Standardized and modular microfluidic platform for fast Lab on Chip system development

  • Apr 22, 2018
  • Sensors and Actuators B: Chemical
  • S Dekker +10
  • Conference Article

A graphical framework for constructing and executing computational networks

  • Jul 01, 2010
  • Christopher L Hall +1
  • PDF
  • Research Article
  • Citations87

Design of multi-scale protein complexes by hierarchical building block fusion

  • Apr 16, 2021
  • Nature Communications
  • Yang Hsia +19
  • Research Article
  • Citations852

3D Soft Metamaterials with Negative Poisson's Ratio

  • Jul 22, 2013
  • Advanced Materials
  • Sahab Babaee +5
  • Research Article
  • Citations1

Next-generation networks application traffic modeling

  • Aug 18, 2008
  • Bell Labs Technical Journal
  • Deirdre H Doherty +2
  • Research Article
  • Citations12

Dynamic system simulation of small satellite projects

  • Jul 09, 2010
  • Acta Astronautica
  • Matthias Raif +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.