Hierarchical master-worker skeletons

  • Authors:

  • Jost Berthold;Mischa Dieterle;Rita Loogen;Steffen Priebe

  • Affiliations:

  • Philipps-Universität Marburg, Fachbereich Mathematik und Informatik, Marburg, Germany;Philipps-Universität Marburg, Fachbereich Mathematik und Informatik, Marburg, Germany;Philipps-Universität Marburg, Fachbereich Mathematik und Informatik, Marburg, Germany;Philipps-Universität Marburg, Fachbereich Mathematik und Informatik, Marburg, Germany

  • Venue:
  • PADL'08 Proceedings of the 10th international conference on Practical aspects of declarative languages
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Master-worker systems are a well-known and often applicable scheme for the parallel evaluation of a pool of tasks, a work pool. The system consists of a master process managing a set of worker processes. After an initial phase with a fixed amount of tasks for each worker, further tasks are distributed in reply to results sent back by the workers. As this setup quickly leads to a bottleneck in the master process, the paper investigates techniques for hierarchically nesting the basic master-worker scheme. We present implementations of hierarchical master-worker skeletons, and how to automatically calculate parameters of the nested skeleton for good performance. Nesting master-worker systems is nontrivial especially in cases where newtasks are dynamically created from previous results (typically breadthordepth-first treesearchalgorithms).Wediscusshowtohandledynamically growing pools in a hierarchy and present a declarative implementation for nested master-worker systems with dynamic task creation. The skeletons are experimentally evaluated with two typical test programs. We analyse their runtime behaviour and the effects of different hierarchies on runtimes via trace visualisations.