Optimization of collective communications in HeteroMPI

Authors:
Alexey Lastovetsky;Maureen O'Flynn;Vladimir Rychkov
Affiliations:
School of Computer Science and Informatics, University College Dublin, Dublin 4, Ireland;School of Computer Science and Informatics, University College Dublin, Dublin 4, Ireland;School of Computer Science and Informatics, University College Dublin, Dublin 4, Ireland
Venue:
PVM/MPI'07 Proceedings of the 14th European conference on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Year:
2007

Citing 8
Cited 4

LogP: towards a realistic model of parallel computation

PPOPP '93 Proceedings of the fourth ACM SIGPLAN symposium on Principles and practice of parallel programming
Efficient Algorithms for the Reduce-Scatter Operation in LogGP

IEEE Transactions on Parallel and Distributed Systems
MagPIe: MPI's collective communication operations for clustered wide area systems

Proceedings of the seventh ACM SIGPLAN symposium on Principles and practice of parallel programming
Automatically tuned collective communications

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
MPI: The Complete Reference

MPI: The Complete Reference
Performance Analysis of MPI Collective Operations

IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Workshop 15 - Volume 16
HeteroMPI: Towards a message-passing library for heterogeneous networks of computers

Journal of Parallel and Distributed Computing
An Accurate Communication Model of a Heterogeneous Cluster Based on a Switch-Enabled Ethernet Network

ICPADS '06 Proceedings of the 12th International Conference on Parallel and Distributed Systems - Volume 2

A Software Tool for Accurate Estimation of Parameters of Heterogeneous Communication Models

Proceedings of the 15th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
MPIBlib: Benchmarking MPI Communications for Parallel Computing on Homogeneous and Heterogeneous Clusters

Proceedings of the 15th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Efficient Collective Communication Paradigms for Hyperspectral Imaging Algorithms Using HeteroMPI

Proceedings of the 15th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Accurate Heterogeneous Communication Models and a Software Tool for Their Efficient Estimation

International Journal of High Performance Computing Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

HeteroMPI is an extension of MPI designed for high performance computing on heterogeneous networks of computers. The recent new feature of HeteroMPI is the optimized version of collective communications. The optimization is based on a novel performance communication model of switch-based computational clusters. In particular, the model reflects significant non-deterministic and non-linear escalations of the execution time of many-to-one collective operations for medium-sized messages. The paper outlines this communication model and describes how HeteroMPI uses this model to optimize one-to-many (scatter-like) and many-to-one (gather-like) communications. We also demonstrate that HeteroMPI collective communications outperform their native counterparts for various MPI implementations and cluster platforms.