CoMPI: Enhancing MPI Based Applications Performance and Scalability Using Run-Time Compression

Authors:
Rosa Filgueira;David E. Singh;Alejandro Calderón;Jesús Carretero
Affiliations:
Department of Computer Science, University Carlos III of Madrid,;Department of Computer Science, University Carlos III of Madrid,;Department of Computer Science, University Carlos III of Madrid,;Department of Computer Science, University Carlos III of Madrid,
Venue:
Proceedings of the 16th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Year:
2009

Citing 8
Cited 3

Dynamic Huffman coding

Journal of Algorithms
COMPASSION: A Parallel I/O Runtime System Including Chunking and Compression for Irregular Applications

HPCN Europe 1998 Proceedings of the International Conference and Exhibition on High-Performance Computing and Networking
MiMPI: A Multithred-Safe Implementation of MPI

Proceedings of the 6th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Runtime Compression of MPI Messanes to Improve the Performance and Scalability of Parallel Applications

Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Fast Lossless Compression of Scientific Floating-Point Data

DCC '06 Proceedings of the Data Compression Conference
Image segmentation based on merging of sub-optimal segmentations

Pattern Recognition Letters
FPC: A High-Speed Compressor for Double-Precision Floating-Point Data

IEEE Transactions on Computers
Issues in developing a thread-safe MPI implementation

EuroPVM/MPI'06 Proceedings of the 13th European PVM/MPI User's Group conference on Recent advances in parallel virtual machine and message passing interface

Transparent neutral element elimination in MPI reduction operations

EuroMPI'10 Proceedings of the 17th European MPI users' group meeting conference on Recent advances in the message passing interface
Adaptive-Compi: Enhancing Mpi-Based Applications - Performance and Scalability by using Adaptive Compression

International Journal of High Performance Computing Applications
An adaptive, scalable, and portable technique for speeding up MPI-based applications

Euro-Par'12 Proceedings of the 18th international conference on Parallel Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents an optimization of MPI communications, called CoMPI , based on run-time compression of MPI messages exchanged by applications. A broad number of compression algorithms have been fully implemented and tested for both MPI collective and point to point primitives. In addition, this paper presents a study of several compression algorithms that can be used for run-time compression, based on the datatype used by applications. This study has been validated by using several MPI benchmarks and real HPC applications. Show that, in most of the cases, using compression reduces the application communication time enhancing application performance and scalability. In this way, CoMPI obtains important improvements in the overall execution time for many of the considered scenarios.