Autonomic query parallelization using non-dedicated computers: an evaluation of adaptivity options

  • Authors:

  • Norman W. Paton;Jorge Buenabad-Chavez;Mengsong Chen;Vijayshankar Raman;Garret Swart;Inderpal Narang;Daniel M. Yellin;Alvaro A. Fernandes

  • Affiliations:

  • School of Computer Science, University of Manchester, Manchester, UK M13 9PL;Department of Computer Science, CINVESTAV-IPN, Mexico, Mexico 07360;School of Computer Science, University of Manchester, Manchester, UK M13 9PL;IBM Almaden Research Center, San Jose, USA 95120;IBM Almaden Research Center, San Jose, USA 95120;IBM Almaden Research Center, San Jose, USA 95120;IBM T.J. Watson Research Labs, Yorktown Heights, USA 10598;School of Computer Science, University of Manchester, Manchester, UK M13 9PL

  • Venue:
  • The VLDB Journal — The International Journal on Very Large Data Bases
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Writing parallel programs that can take advantage of non-dedicated processors is much more difficult than writing such programs for networks of dedicated processors. In a non-dedicated environment such programs must use autonomic techniques to respond to the unpredictable load fluctuations that prevail in the computational environment. In adaptive query processing (AQP), several techniques have been proposed for dynamically redistributing processor load assignments throughout a computation to take account of varying resource capabilities, but we know of no previous study that compares their performance. This paper presents a simulation-based evaluation of these autonomic parallelization techniques in a uniform environment and compares how well they improve the performance of the computation. Four published strategies are compared with a new algorithm that seeks to overcome some weaknesses identified in the existing approaches. In addition, we explore the use of techniques from online algorithms to provide a firm foundation for determining when to adapt in two of the existing algorithms. The evaluations identify situations in which each strategy may be used effectively and in which it should be avoided.