An investigation into the performance of reduction algorithms under load imbalance

Authors:
Petar Marendi$#263;Jan Lemeire;Tom Haber;Dean Vučini$#263;Peter Schelkens
Affiliations:
ETRO Dept., Vrije Universiteit Brussel (VUB), Brussels, Belgium, FMI Dept., Interdisciplinary Institute for Broadband Technology (IBBT), Ghent, Belgium;ETRO Dept., Vrije Universiteit Brussel (VUB), Brussels, Belgium, FMI Dept., Interdisciplinary Institute for Broadband Technology (IBBT), Ghent, Belgium;EDM, UHasselt, Diepenbeek, Belgium;ETRO Dept., Vrije Universiteit Brussel (VUB), Brussels, Belgium, FMI Dept., Interdisciplinary Institute for Broadband Technology (IBBT), Ghent, Belgium;ETRO Dept., Vrije Universiteit Brussel (VUB), Brussels, Belgium, FMI Dept., Interdisciplinary Institute for Broadband Technology (IBBT), Ghent, Belgium
Venue:
Euro-Par'12 Proceedings of the 18th international conference on Parallel Processing
Year:
2012

Citing 4
Cited 0

Introduction to parallel computing: design and analysis of algorithms

Introduction to parallel computing: design and analysis of algorithms
On Benchmarking Collective MPI Operations

Proceedings of the 9th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Massively parallel volume rendering using 2-3 swap image compositing

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Accurately measuring overhead, communication time and progression of blocking and nonblocking collective operations at massive scale

International Journal of Parallel, Emergent and Distributed Systems - Performance evaluation of ubiquitous computing and networked systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Today, most reduction algorithms are optimized for balanced workloads; they assume all processes will start the reduction at about the same time. However, in practice this is not always the case and significant load imbalances may occur and affect the performance of said algorithms. In this paper we investigate the impact of such imbalances on the most commonly employed reduction algorithms and propose a new algorithm specifically adapted to the presented context. Firstly, we analyze the optimistic case where we have a priori knowledge of all imbalances and propose a near-optimal solution. In the general case, where we do not have any foreknowledge of the imbalances, we propose a dynamically rebalanced tree reduction algorithm. We show experimentally that this algorithm performs better than the default OpenMPI and MVAPICH2 implementations.