Experiences using hybrid MPI/OpenMP in the real world: Parallelization of a 3D CFD solver for multi-core node clusters

  • Authors:

  • Gabriele Jost;Bob Robins

  • Affiliations:

  • (Correspd. Tel.: +1 831 656 3321/ E-mail: gjost@tacc.utexas.edu) Texas Advanced Computing Center, The University of Texas, Austin, TX, USA;NorthWest Research Associates, Inc., Bellevue, WA, USA

  • Venue:
  • Scientific Programming - Exploring Languages for Expressing Medium to Massive On-Chip Parallelism
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Today most systems in high-performance computing (HPC) feature a hierarchical hardware design: shared-memory nodes with several multi-core CPUs are connected via a network infrastructure. When parallelizing an application for these architectures it seems natural to employ a hierarchical programming model such as combining MPI and OpenMP. Nevertheless, there is the general lore that pure MPI outperforms the hybrid MPI/OpenMP approach. In this paper, we describe the hybrid MPI/OpenMP parallelization of IR3D (Incompressible Realistic 3-D) code, a full-scale real-world application, which simulates the environmental effects on the evolution of vortices trailing behind control surfaces of underwater vehicles. We discuss performance, scalability and limitations of the pure MPI version of the code on a variety of hardware platforms and show how the hybrid approach can help to overcome certain limitations.