A Fast Algorithm for Online Placement and Reorganization of Replicated Data

  • Authors:
  • R. J. Honicky;Ethan L. Miller

  • Affiliations:
  • -;-

  • Venue:
  • IPDPS '03 Proceedings of the 17th International Symposium on Parallel and Distributed Processing
  • Year:
  • 2003

Quantified Score

Hi-index 0.01

Visualization

Abstract

As storage systems scale to thousands of disks, data distribution and load balancing become increasingly important. We present an algorithm for allocating data objects to disks as a system as it grows from a few disks to hundreds or thousands. A client using our algorithm can locate a data object in microseconds without consulting a central server or maintaining a full mapping of objects or buckets to disks. Despite requiring little global configuration data, ouralgorithm is probabilistically optimal in both distributing data evenly and minimizing data movement when new storage is added to the system. Moreover, our algorithm supports weighted allocation and variable levels of object replication, both of which are needed to permit systems to efficiently grow while accommodating new technology.