A framework for characterizing overlap of communication and computation in parallel applications

Authors:
Aniruddha G. Shet;P. Sadayappan;David E. Bernholdt;Jarek Nieplocha;Vinod Tipparaju
Affiliations:
The Ohio State University, Columbus, USA 43210;The Ohio State University, Columbus, USA 43210;Oak Ridge National Laboratory, Oak Ridge, USA 37831;Pacific Northwest National Laboratory, Richland, USA 99352;Pacific Northwest National Laboratory, Richland, USA 99352
Venue:
Cluster Computing
Year:
2008

Citing 12
Cited 0

Performance analysis of distributed applications using automatic classification of communication inefficiencies

Proceedings of the 14th international conference on Supercomputing
Statistical scalability analysis of communication operations in distributed applications

PPoPP '01 Proceedings of the eighth ACM SIGPLAN symposium on Principles and practices of parallel programming
Dynamic statistical profiling of communication activity in distributed applications

SIGMETRICS '02 Proceedings of the 2002 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
User-Level Network Interface Protocols

Computer
Myrinet: A Gigabit-per-Second Local Area Network

IEEE Micro
COMB: A Portable Benchmark Suite for Assessing MPI Overlap

CLUSTER '02 Proceedings of the IEEE International Conference on Cluster Computing
The Quadrics Network (QsNet): High-Performance Clustering Technology

HOTI '01 Proceedings of the The Ninth Symposium on High Performance Interconnects
Overlapping of communication and computation and early binding: fundamental mechanisms for improving parallel performance on clusters of workstations

Overlapping of communication and computation and early binding: fundamental mechanisms for improving parallel performance on clusters of workstations
An analysis of the impact of MPI overlap and independent progress

Proceedings of the 18th annual international conference on Supercomputing
RDMA read based rendezvous protocol for MPI over InfiniBand: design alternatives and benefits

Proceedings of the eleventh ACM SIGPLAN symposium on Principles and practice of parallel programming
High Performance Remote Memory Access Communication: The Armci Approach

International Journal of High Performance Computing Applications
Optimisation and performance evaluation of mechanisms for latency tolerance in remote memory access communication on clusters

International Journal of High Performance Computing and Networking

Quantified Score

Hi-index	0.00

Visualization

Abstract

Effective overlap of computation and communication is a well understood technique for latency hiding and can yield significant performance gains for applications on high-end computers. In this paper, we propose an instrumentation framework for message-passing systems to characterize the degree of overlap of communication with computation in the execution of parallel applications. The inability to obtain precise time-stamps for pertinent communication events is a significant problem, and is addressed by generation of minimum and maximum bounds on achieved overlap. The overlap measures can aid application developers and system designers in investigating scalability issues. The approach has been used to instrument two MPI implementations as well as the ARMCI system. The implementation resides entirely within the communication library and thus integrates well with existing approaches that operate outside the library. The utility of the framework is demonstrated by analyzing communication-computation overlap for micro-benchmarks and the NAS benchmarks, and the insights obtained are used to modify the NAS SP benchmark, resulting in improved overlap.