Transparent adaptive parallelism on NOWs using OpenMP

  • Authors:

  • Alex Scherer;Honghui Lu;Thomas Gross;Willy Zwaenepoel

  • Affiliations:

  • Departement Informatik, ETH Zürich, CH 8092 Zürich;Department of Electrical and Computer Engineering, Rice University, Houston, TX;Departement Informatik, ETH Zürich, CH 8092 Zürich and School of Computer Science, Carnegie Mellon University, Pittsburgh, PA;Department of Computer Science, Rice University, Houston, TX

  • Venue:
  • Proceedings of the seventh ACM SIGPLAN symposium on Principles and practice of parallel programming
  • Year:
  • 1999

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a system that allows OpenMP programs to execute on a network of workstations with a variable number of nodes. The ability to adapt to a variable number of nodes allows a program to take advantage of additional nodes that become available after it starts execution, or to gracefully scale down when the number of available nodes is reduced. We demonstrate that the cost of adaptation is modest; the system allows a program to adapt at a moderate rate without much performance loss.Two ideas underlie the efficiency of our design. First, we recognize that OpenMP programs exhibit convenient adaptation points during their execution, points at which the cost of adaptation can be much reduced. Second, by allowing a process a certain grace period before it must leave a node, we insure that most adaptations can occur at these adaptation points, and thus at low cost. Migration of a process, a much more expensive method for providing adaptivity, is used only as a back-up solution, when the process cannot reach an adaptation point within the grace period.Our implementation consists of an OpenMP pre-processor that generates TreadMarks distributed shared memory (DSM) programs, and a version of TreadMarks modified to adapt to a variable number of nodes. Using a DSM as the underlying substrate facilitates the data (re-)distribution necessary after an adaptation.