An adaptive, scalable, and portable technique for speeding up MPI-based applications

Authors:
Rosa Filgueira;Malcolm Atkinson;Alberto Nuñ/ez;Javier Fern$#225/ndez
Affiliations:
School of Informatics, University of Edinburgh, Edinburgh, U.K.;School of Informatics, University of Edinburgh, Edinburgh, U.K.;Dept. Sistemas Inform$#225/ticos y Computació/n, University Complutense de Madrid, Madrid, Spain;Dept. Arquitectura de Computadores, University Carlos III de Madrid, Legané/s, Spain
Venue:
Euro-Par'12 Proceedings of the 18th international conference on Parallel Processing
Year:
2012

Citing 11
Cited 0

Dynamic Huffman coding

Journal of Algorithms
A high-performance, portable implementation of the MPI message passing interface standard

Parallel Computing
COMPASSION: A Parallel I/O Runtime System Including Chunking and Compression for Irregular Applications

HPCN Europe 1998 Proceedings of the International Conference and Exhibition on High-Performance Computing and Networking
MiMPI: A Multithred-Safe Implementation of MPI

Proceedings of the 6th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Runtime Compression of MPI Messanes to Improve the Performance and Scalability of Parallel Applications

Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Toward Scalable Performance Visualization with Jumpshot

International Journal of High Performance Computing Applications
Fast Lossless Compression of Scientific Floating-Point Data

DCC '06 Proceedings of the Data Compression Conference
A Flexible and Dynamic Infrastructure for MPI Tool Interoperability

ICPP '06 Proceedings of the 2006 International Conference on Parallel Processing
FPC: A High-Speed Compressor for Double-Precision Floating-Point Data

IEEE Transactions on Computers
CoMPI: Enhancing MPI Based Applications Performance and Scalability Using Run-Time Compression

Proceedings of the 16th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Adaptive-Compi: Enhancing Mpi-Based Applications - Performance and Scalability by using Adaptive Compression

International Journal of High Performance Computing Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a portable optimization for MPI communications, called PRAcTICaL-MPI (Portable Adaptive Compression Library- MPI). PRAcTICaL-MPI reduces the data volume exchanged among processes by using lossless compression and offers two main advantages. Firstly, it is independent of the MPI implementation and the application used. Secondly, it allows for turning the compression on and off and selecting the most appropriate compression algorithm at run-time, depending on the characteristics of each message and on network performance. We have validated PRAcTICaL-MPI in different MPI implementations and HPC clusters. The evaluation shows that compressing MPI messages with the best algorithm and only when it is worthwhile, we obtain a great reduction in the overall execution time for many of the scenarios considered.