A case for using MPI's derived datatypes to improve I/O performance
SC '98 Proceedings of the 1998 ACM/IEEE conference on Supercomputing
MPI-The Complete Reference, Volume 1: The MPI Core
MPI-The Complete Reference, Volume 1: The MPI Core
Learning from the Success of MPI
HiPC '01 Proceedings of the 8th International Conference on High Performance Computing
Fault Tolerant MPI for the HARNESS Meta-computing System
ICCS '01 Proceedings of the International Conference on Computational Sciences-Part I
Exploiting Transparent Remote Memory Access for Non-Contiguous- and One-Sided-Communication
IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
A Benchmark for MPI Derived Datatypes
Proceedings of the 7th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
The Implementation of One-Sided Communications for WMPI II
Proceedings of the 8th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Constructing MPI Input-output Datatypes for Efficient Transpacking
Proceedings of the 15th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Introduction to Acceleration for MPI Derived Datatypes Using an Enhancer of Memory and Network
Proceedings of the 15th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Processing MPI Datatypes Outside MPI
Proceedings of the 16th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Exploiting Efficient Transpacking for One-Sided Communication and MPI-IO
Proceedings of the 16th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
MPI datatype marshalling: a case study in datatype equivalence
EuroMPI'10 Proceedings of the 17th European MPI users' group meeting conference on Recent advances in the message passing interface
Parallel zero-copy algorithms for fast Fourier transform and conjugate gradient using MPI datatypes
EuroMPI'10 Proceedings of the 17th European MPI users' group meeting conference on Recent advances in the message passing interface
Using MPI derived datatypes in numerical libraries
EuroMPI'11 Proceedings of the 18th European MPI Users' Group conference on Recent advances in the message passing interface
Performance expectations and guidelines for MPI derived datatypes
EuroMPI'11 Proceedings of the 18th European MPI Users' Group conference on Recent advances in the message passing interface
Automatic memory optimizations for improving MPI derived datatype performance
EuroPVM/MPI'06 Proceedings of the 13th European PVM/MPI User's Group conference on Recent advances in parallel virtual machine and message passing interface
Can MPI be used for persistent parallel services?
EuroPVM/MPI'06 Proceedings of the 13th European PVM/MPI User's Group conference on Recent advances in parallel virtual machine and message passing interface
Open issues in MPI implementation
ACSAC'07 Proceedings of the 12th Asia-Pacific conference on Advances in Computer Systems Architecture
MPI 3 and beyond: why MPI is successful and what challenges it faces
EuroMPI'12 Proceedings of the 19th European conference on Recent Advances in the Message Passing Interface
Micro-applications for communication data access patterns and MPI datatypes
EuroMPI'12 Proceedings of the 19th European conference on Recent Advances in the Message Passing Interface
MPI datatype processing using runtime compilation
Proceedings of the 20th European MPI Users' Group Meeting
| Hi-index | 0.00 |
The Message Passing Interface (MPI) incorporates a mechanism for describing structured, non-contiguous memory layouts for use as communication buffers in MPI communication functions. The rationale behind the derived datatype mechanism is to alleviate the user from tedious packing and unpacking of non-consecutive data into contiguous communication buffers. Furthermore, the mechanism makes it possible to improve performance by saving on internal buffering. Apparently, current MPI implementations entail considerable performance penalties when working with derived datatypes. We describe a new method called flattening on the fly for the efficient handling of derived datatypes in MPI. The method aims at exploiting regularities in the memory layout described by the datatype as far as possible. In addition it considerably reduces the overhead for parsing the datatype. Flattening on the fly has been implemented and evaluated on an NEC SX-4 vector supercomputer. On the SX-4 flattening on the fly performs significantly better than previous methods, resulting in performance comparable to what the user can in the best case achieve by packing and unpacking data manually. Also on a PC cluster the method gives worthwhile improvements in cases that are not handled well by the conventional implementation.