Adaptive workload allocation in query processing in autonomous heterogeneous environments

  • Authors:
  • Anastasios Gounaris;Jim Smith;Norman W. Paton;Rizos Sakellariou;Alvaro A. Fernandes;Paul Watson

  • Affiliations:
  • Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece 541 24;School of Computing Science, University of Newcastle upon Tyne, Newcastle upon Tyne, UK NE1 7RU;School of Computer Science, University of Manchester, Manchester, UK M13 9PL;School of Computer Science, University of Manchester, Manchester, UK M13 9PL;School of Computer Science, University of Manchester, Manchester, UK M13 9PL;School of Computing Science, University of Newcastle upon Tyne, Newcastle upon Tyne, UK NE1 7RU

  • Venue:
  • Distributed and Parallel Databases
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

The increasing prevalence of networked storage and computational resources, along with middleware for managing resource access and sharing, raises the prospect that queries can be run over resources obtained on demand, rather than on dedicated infrastructures. However, the movement of query processing into non-dedicated environments means that it is necessary to take account of the partial information and unstable conditions that characterise autonomous, shared, distributed settings. Thus, query processing on grid platforms needs to be adaptive, revising evaluation strategies at query runtime in response to the evolving environment, such as changes to machine load and availability. To address this challenge, adaptive techniques are described that: (i) balance load across plan partitions supporting intra-operator parallelism; (ii) remove bottlenecks in pipelined plans supporting inter-operator parallelism; and (iii) combine the two aforementioned techniques. The approach has been empirically evaluated in a grid-enabled adaptive query processor.