Parity logging with reserved space: towards efficient updates and recovery in erasure-coded clustered storage

  • Authors:

  • Jeremy C. W. Chan;Qian Ding;Patrick P. C. Lee;Helen H. W. Chan

  • Affiliations:

  • The Chinese University of Hong Kong;The Chinese University of Hong Kong;The Chinese University of Hong Kong;The Chinese University of Hong Kong

  • Venue:
  • FAST'14 Proceedings of the 12th USENIX conference on File and Storage Technologies
  • Year:
  • 2014

Quantified Score

Hi-index 0.00

Visualization

Abstract

Many modern storage systems adopt erasure coding to provide data availability guarantees with low redundancy. Log-based storage is often used to append new data rather than overwrite existing data so as to achieve high update efficiency, but introduces significant I/O overhead during recovery due to reassembling updates from data and parity chunks. We propose parity logging with reserved space, which comprises two key design features: (1) it takes a hybrid of in-place data updates and log-based parity updates to balance the costs of updates and recovery, and (2) it keeps parity updates in a reserved space next to the parity chunk to mitigate disk seeks. We further propose a workload-aware scheme to dynamically predict and adjust the reserved space size. We prototype an erasure-coded clustered storage system called CodFS, and conduct testbed experiments on different update schemes under synthetic and real-world workloads. We show that our proposed update scheme achieves high update and recovery performance, which cannot be simultaneously achieved by pure in-place or log-based update schemes.