Data partitioning and load balancing in parallel disk systems

  • Authors:
  • Peter Scheuermann;Gerhard Weikum;Peter Zabback

  • Affiliations:
  • Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL 60208, USA/ E-mail: peters@eecs.nwu.edu;Department of Computer Science, University of the Saarland, P.O. Box 151150, D-66041 Saarbrü/cken, Germany/ E-mail: weikum@cs.uni-sb.de;Tandem Computers Incorporated, 10100 North Tantau Avenue, Cupertino, CA 95014-2542, USA/ E-mail: zabback@loc251.tandem.com

  • Venue:
  • The VLDB Journal — The International Journal on Very Large Data Bases
  • Year:
  • 1998

Quantified Score

Hi-index 0.00

Visualization

Abstract

Parallel disk systems provide opportunities for exploiting I/O parallelism in two possible ways, namely via inter-request and intra-request parallelism. In this paper, we discuss the main issues in performance tuning of such systems, namely striping and load balancing, and show their relationship to response time and throughput. We outline the main components of an intelligent, self-reliant file system that aims to optimize striping by taking into account the requirements of the applications, and performs load balancing by judicious file allocation and dynamic redistributions of the data when access patterns change. Our system uses simple but effective heuristics that incur only little overhead. We present performance experiments based on synthetic workloads and real-life traces.