Efficient layering for high speed communication: the MPI over Fast Messages (FM) experience

  • Authors:

  • Mario Lauria;Scott Pakin;Andrew Chien

  • Affiliations:

  • Department of Computer Science and Engineering, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0114, USA;Department of Computer Science, University of Illinois at Urbana-Champaign, 1304 W. Springfield Avenue, Urbana, IL 61801, USA;Science Applications International Corporation Chair Professor, Department of Computer Science and Engineering University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0114, US ...

  • Venue:
  • Cluster Computing
  • Year:
  • 1999

Quantified Score

Hi-index 0.00

Visualization

Abstract

We describe our experience of designing, implementing, and evaluating two generations of high performance communication libraries, Fast Messages (FM) for Myrinet. In FM 1, we designed a simple interface and provided guarantees of reliable and in-order delivery, and flow control. While this was a significant improvement over previous systems, it was not enough. Layering MPI atop FM 1 showed that only about 35% of the FM 1 bandwidth could be delivered to higher level communication APIs. Our second generation communication layer, FM 2, addresses the identified problems, providing gather-scatter, interlayer scheduling, receiver flow control, as well as some convenient API features which simplify programming. FM 2 can deliver 55–95% to higher level APIs such as MPI. This is especially impressive as the absolute bandwidths delivered have increased over fourfold to 90 MB/s. We describe general issues encountered in matching two communication layers, and our solutions as embodied in FM 2.