Benefit of Limited Time Sharing in the Presence of Very Large Parallel Jobs

  • Authors:
  • Su-Hui Chiang;Chuyong Fu

  • Affiliations:
  • Portland State University, Oregon;Portland State University, Oregon

  • Venue:
  • IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper provides a comprehensive performance evaluation of job scheduling policies for parallel systems on which large jobs require the maximum or close to the maximum resource available on the system. A wide range of policies are evaluated, including nonpreemptive Backfill policies, time-sharing with Gang Scheduling, and dynamic Equi-spatial policies. Through detailed performance analysis, key problems of each class of policies are identified. As a simpler alternative to Gang Scheduling and Equi-spatial, we propose using a short runtime limit (e.g., one hour as opposed to 10's hours) for Backfill policies. Our simulation results show that applying a short runtime limit on jobs that request a sufficiently large number of processors has the potential to significantly improve FCFS-Backfill policies for all job classes in most workloads studied.