Xen and the Art of Cluster Scheduling

  • Authors:

  • Niels Fallenbeck;Hans-Joachim Picht;Matthew Smith;Bernd Freisleben

  • Affiliations:

  • University of Marburg;University of Marburg;University of Marburg;University of Marburg

  • Venue:
  • VTDC '06 Proceedings of the 2nd International Workshop on Virtualization Technology in Distributed Computing
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

In shared use clusters, scheduling systems must schedule both serial and parallel jobs in a fair manner, while at the same time optimizing overall cluster efficiency. Since serial and parallel jobs conflict considerably, scheduling both types of jobs concurrently is a difficult task. Two major strategies are in common use: partitioning the cluster (thus avoiding the problem) and reservation combined with backfilling. Both have major drawbacks in overall performance, ease of use and fairness depending on the particular configuration, which can lead to heated debates between the users of the different types of jobs. In this paper, we introduce an approach to create dynamic virtual cluster partitions using para-virtualization techniques, to deal with the conflicts between parallel and serial jobs. The system dynamically adjusts to different types of job loads and offers easy and transparent use and configuration to both users and administrators, without resorting to complicated runtime prediction or backfilling algorithms. A proof-of-concept implementation based on the Sun Grid Engine scheduling system and Xen is presented.