Speeding up distributed request-response workflows

  • Authors:

  • Virajith Jalaparti;Peter Bodik;Srikanth Kandula;Ishai Menache;Mikhail Rybalkin;Chenyu Yan

  • Affiliations:

  • University of Illinois at Urbana-Champaign, Champaign, IL, USA;Microsoft Research, Redmond, WA, USA;Microsoft Research, Redmond, WA, USA;Microsoft Research, Redmond, WA, USA;St. Petersburg Department of Steklov Institute of Mathematics of Russian Academy of Sciences, St. Petersburg, Russian Fed.;Microsoft Bing, Bellevue, WA, USA

  • Venue:
  • Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

We found that interactive services at Bing have highly variable datacenter-side processing latencies because their processing consists of many sequential stages, parallelization across 10s-1000s of servers and aggregation of responses across the network. To improve the tail latency of such services, we use a few building blocks: reissuing laggards elsewhere in the cluster, new policies to return incomplete results and speeding up laggards by giving them more resources. Combining these building blocks to reduce the overall latency is non-trivial because for the same amount of resource (e.g., number of reissues), different stages improve their latency by different amounts. We present Kwiken, a framework that takes an end-to-end view of latency improvements and costs. It decomposes the problem of minimizing latency over a general processing DAG into a manageable optimization over individual stages. Through simulations with production traces, we show sizable gains; the 99th percentile of latency improves by over 50% when just 0.1% of the responses are allowed to have partial results and by over 40% for 25% of the services when just 5% extra resources are used for reissues.