An adaptive load balancing algorithm for large data parallel processing with communication delay

Authors:
Kenji Imasaki;Jemal H. Abawajy
Affiliations:
Carleton University, Ottawa, Canada;Carleton University, Ottawa, Canada
Venue:
ICCS'03 Proceedings of the 2003 international conference on Computational science
Year:
2003

Citing 7
Cited 0

A performance evaluation of four parallel join algorithms in a shared-nothing multiprocessor environment

SIGMOD '89 Proceedings of the 1989 ACM SIGMOD international conference on Management of data
Performance of load balancing techniques for join operations in shared-noting database management systems

Journal of Parallel and Distributed Computing
Eddies: continuously adaptive query processing

SIGMOD '00 Proceedings of the 2000 ACM SIGMOD international conference on Management of data
Scrambling query plans to cope with unexpected delays

DIS '96 Proceedings of the fourth international conference on on Parallel and distributed information systems
Performance Comparison of Pipelined Hash Joins on Workstation Clusters

HiPC '02 Proceedings of the 9th International Conference on High Performance Computing
An Adaptive Hash Join Algorithm on a Network of Workstations

IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Performance Evaluation of Nested-Loop Join Processing on Networks of Workstations

ICPADS '00 Proceedings of the Seventh International Conference on Parallel and Distributed Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Achieving good load balance in cluster environment is difficult due to its dynamic nature. Besides, the data to be processed may be transfered from geographically distant sites at a fluctuating transfer rate. This paper proposes a load balancing algorithm to deal with such fluctuating transfer rate. The novelty of the algorithm is to use buddy concept in processing node grouping and the data distribution statistics obtained at the source sites. We use single join processing in database query processing as its application.