Design and Implementation of Open MPI over Quadrics/Elan4

  • Authors:
  • Weikuan Yu;Tim S. Woodall;Rich L. Graham;Dhabaleswar K. Panda

  • Affiliations:
  • The Ohio State University;Los Alamos National Laboratory;Los Alamos National Laboratory;The Ohio State University

  • Venue:
  • IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Open MPI is a project recently initiated to provide a fault-tolerant, multi-network capable implementation of MPI-2, based on experiences gained from FT-MPI, LA-MPI, LAM/MPI, and MVAPICH projects. Its initial communication architecture is layered on top of TCP/IP. In this paper, we have designed and implemented Open MPI point-to-point layer on top of a high-end interconnect, Quadrics/Elan4. The restriction of Quadrics static process model has been overcome to accommodate the requirement of MPI-2 dynamic process management. Quadrics Queued-based Direct Memory Access (QDMA) and Remote Direct Memory Access (RDMA) mechanisms have been integrated to form a low-overhead, high-performance transport layer. Lightweight asynchronous progress is made possible with a combination of Quadrics chained event and QDMA mechanisms. Experimental results indicate that the resulting point-to-point transport layer is able to achieve comparable performance to Quadrics native QDMA operations, from which it is derived. Our implementation provides an MPI-2 compliant message passing library over Quadrics/Elan4 with a performance comparable to MPICH-Quadrics.