High-throughput low-latency fine-grained disk logging

  • Authors:

  • Dilip Nijagal Simha;Tzi-cker Chiueh;Ganesh Karuppur Rajagopalan;Pallav Bose

  • Affiliations:

  • Stony Brook University & Industrial Technology Research Institute, Stony Brook, NY, USA;Stony Brook University & Industrial Technology Research Institute, Hsinchu, Taiwan Roc;Stony Brook University, Stony Brook, NY, USA;Stony Brook University, Stony Brook, NY, USA

  • Venue:
  • Proceedings of the ACM SIGMETRICS/international conference on Measurement and modeling of computer systems
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Synchronously logging updates to persistent storage first and then asynchronously committing these updates to their rightful storage locations is a well-known and heavily used technique to improve the sustained throughput of write-intensive disk-based data processing systems, whose latency and throughput accordingly are largely determined by the latency and throughput of the underlying logging mechanism. The conventional wisdom is that logging operations are relatively straightforward to optimize because the associated disk access pattern is largely sequential. However, it turns out that to achieve both high throughput and low latency for fine-grained logging operations, whose payload size is smaller than a disk sector, is extremely challenging. This paper describes the experiences and lessons we have gained from building a disk logging system that can successfully deliver over 1.2 million 256-byte logging operations per second, with the average logging latency below 1 msec.