AnthillSched: a scheduling strategy for irregular and iterative I/O-intensive parallel jobs

  • Authors:

  • Luís Fabrício Góes;Pedro Guerra;Bruno Coutinho;Leonardo Rocha;Wagner Meira;Renato Ferreira;Dorgival Guedes;Walfredo Cirne

  • Affiliations:

  • Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brasil;Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brasil;Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brasil;Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brasil;Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brasil;Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brasil;Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brasil;Universidade Federal de Campina Grande, Campina Grande, PB

  • Venue:
  • JSSPP'05 Proceedings of the 11th international conference on Job Scheduling Strategies for Parallel Processing
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Irregular and iterative I/O-intensive jobs need a different approach from parallel job schedulers. The focus in this case is not only the processing requirements anymore: memory, network and storage capacity must all be considered in making a scheduling decision. Job executions are irregular and data dependent, alternating between CPU-bound and I/O-bound phases. In this paper, we propose and implement a parallel job scheduling strategy for such jobs, called AnthillSched, based on a simple heuristic: we map the behavior of a parallel application with minimal resources as we vary its input parameters. From that mapping we infer the best scheduling for a certain set of input parameters given the available resources. To test and verify AnthillSched we used logs obtained from a real system executing data mining jobs. Our main contributions are the implementation of a parallel job scheduling strategy in a real system and the performance analysis of AnthillSched, which allowed us to discard some other scheduling alternatives considered previously.