Providing tunable consistency for a parallel file store

  • Authors:
  • Murali Vilayannur;Partho Nath;Anand Sivasubramaniam

  • Affiliations:
  • Department of Computer Science and Engineering, Pennsylvania State University;Department of Computer Science and Engineering, Pennsylvania State University;Department of Computer Science and Engineering, Pennsylvania State University

  • Venue:
  • FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Consistency, throughput, and scalability form the backbone of a cluster-based parallel file system. With little or no information about the workloads to be supported, a file system designer has to often make a one-glove-fits-all decision regarding the consistency policies. Taking a hard stance on consistency demotes throughput and scalability to second-class status, having to make do with whatever leeway is available. Leaving the choice and granularity of consistency policies to the user at open/mount time provides an attractive way of providing the best of all worlds. We present the design and implementation of such a file-store, CAPFS (Content Addressable Parallel File System), that allows the user to define consistency semantic policies at runtime. A client-side plug-in architecture based on user-written plug-ins leaves the choice of consistency policies to the end-user. The parallelism exploited by use of multiple data stores provides for bandwidth and scalability. We provide extensive evaluations of our prototype file system on a concurrent read/write workload and a parallel tiled visualization code.