Exploiting replication and data reuse to efficiently schedule data-intensive applications on grids

  • Authors:

  • Elizeu Santos-Neto;Walfredo Cirne;Francisco Brasileiro;Aliandro Lima

  • Affiliations:

  • Universidade Federal de Campina Grande;Universidade Federal de Campina Grande;Universidade Federal de Campina Grande;Universidade Federal de Campina Grande

  • Venue:
  • JSSPP'04 Proceedings of the 10th international conference on Job Scheduling Strategies for Parallel Processing
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

Data-intensive applications executing over a computational grid demand large data transfers. These are costly operations. Therefore, taking them into account is mandatory to achieve efficient scheduling of data-intensive applications on grids. Further, within a heterogeneous and ever changing environment such as a grid, better schedules are typically attained by heuristics that use dynamic information about the grid and the applications. However, this information is often difficult to be accurately obtained. On the other hand, although there are schedulers that attain good performance without requiring dynamic information, they were not designed to take data transfer into account. This paper presents Storage Affinity, a novel scheduling heuristic for bag-of-tasks data-intensive applications running on grid environments. Storage Affinity exploits a data reuse pattern, common on many data-intensive applications, that allows it to take data transfer delays into account and reduce the makespan of the application. Further, it uses a replication strategy that yields efficient schedules without relying upon dynamic information that is difficult to obtain. Our results show that Storage Affinity may attain better performance than the state-of-the-art knowledge-dependent schedulers. This is achieved at the expense of consuming more CPU cycles and network bandwidth.