On/off-line prediction applied to job scheduling on non-dedicated NOWs

  • Authors:

  • Mauricio Hanzich;Porfidio Hernández;Francesc Giné;Francese Solsona;Josep L. Lérida

  • Affiliations:

  • Department of Computer Application in Science and Engineering, Barcelona Supercomputing Center, Barcelona, Spain;Department of Computer Architecture & Operating Systems, Autonomous University of Barcelona, Barcelona, Spain;Department of Computer Science, University of Lleida, Lleida, Spain;Department of Computer Science, University of Lleida, Lleida, Spain;Department of Computer Science, University of Lleida, Lleida, Spain

  • Venue:
  • Journal of Computer Science and Technology - Special issue on natural language processing
  • Year:
  • 2011

Quantified Score

Hi-index 0.01

Visualization

Abstract

This paper proposes a prediction engine designed for non-dedicated clusters, which is able to estimate the turnaround time for parallel applications, even in the presence of serial workload of the workstation owner. The prediction engine can be configured to work with three different estimation kernels: a Historical kernel, a Simulation kernel based on analytical models and an integration of both, named Hybrid kernel. These estimation proposals were integrated into a scheduling system, named CISNE, which can be executed in an on-line or off-line mode. The accuracy of the proposed estimation methods was evaluated in relation to different job scheduling policies in a real and a simulated cluster environment. In both environments, we observed that the Hybrid system gives the best results because it combines the ability of a simulation engine to capture the dynamism of a non-dedicated environment together with the accuracy of the historical methods to estimate the application runtime considering the state of the resources.