Varying the domain size of the dynamic load-balancing algorithm DASUD for SPMD and MPMD programming scenarios

• Authors: A. Cortes;A. Ripoll;M. A. Senar;E. Luque
• Venue: International Journal of High Performance Computing and Networking
• Year: 2004

Dynamic load balancing is a key problem for the efficient use ofparallel systems when solving applications with unpredictable loadestimates. However, depending on the underlying programmingparadigm Single Program Multiple Data (SPMD) or Multiple ProgramMultiple Data (MPMD) the balancing requirements vary. In SPMDscenarios, a perfect load balance is desired, whereas in MPMDscenarios it might be better to quickly obtain a large reduction inload imbalance in a short period of time. We propose extending thelocal domain of a given processor in the load-balancing algorithmsto find a better scope for each paradigm. For that purpose, wepresent a generalised version of the Diffusion Algorithm SearchingUnbalanced Domains (called ds-DASUD), which extends thelocal domain of each processor beyond its immediate neighbour.ds-DASUD belongs to the iterative distributedload-balancing (IDLB) class and, in its original formulation,operates in a diffusion scheme where a processor balances its loadwith all its immediate neighbours (ds=1). We evaluatethis algorithm for the two programming paradigms varying the domainsize. The evaluation was carried out using two simulators(load-balancing and network simulators) for a large set of loaddistributions that exhibit different degrees of initial workloadunbalancing. These distributions were applied to torus andhypercube topologies, and the number of processors ranged from 8 to128. From these experiments, we conclude that the 1-DASUD fits wellfor SPMD scenarios, whereas for MPMD 3-DASUD and ((d/2)+1)-DASUDfor hypercube and torus topologies, respectively, obtain the besttrade-off between the imbalance reduction (up to 85%) and the costincurred in reaching it.