Distributed file organization with scalable cost/performance

  • Authors:

  • Radek Vingralek;Yuri Breitbart;Gerhard Weikum

  • Affiliations:

  • Department of Computer Science, ETH Zurich, CH-8092, Switzerland and Department of Computer Science, University of Kentucky, Lexington, KY;Department of Computer Science, ETH Zurich, CH-8092, Switzerland and Department of Computer Science, University of Kentucky, Lexington, KY;Department of Computer Science, ETH Zurich, CH-8092, Switzerland

  • Venue:
  • SIGMOD '94 Proceedings of the 1994 ACM SIGMOD international conference on Management of data
  • Year:
  • 1994

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper presents a distributed file organization for record-structured, disk-resident files with key-based exact-match access. The file is organized into buckets that are spread across multiple servers, where a server may hold multiple buckets. Client requests are serviced by mapping keys onto buckets and looking up the corresponding server in an address table. Dynamic growth in terms of file size and access load is supported by bucket splits and migration onto other existing or newly acquired servers.The significant and challenging problem addressed here is how to achieve scalability so that both the file size and the client throughput can be scaled up by linearly increasing the number of servers and dynamically redistributing data. Unlike previous work with similar objectives, our data redistribution considers explicitly the cost/performance ratio of the system by aiming to minimize the number of servers that are acquired to provide the required performance. A new server is acquired only if the overall server utilization in the system does not drop below a specified threshold. Preliminary simulation results show that the goal of scalability with controlled cost/performance is indeed achieved to a large extent.