Effective seamless remote MPI-I/O operations with derived data types using PVFS2

Authors:
Yuichi Tsujita
Affiliations:
Department of Electronic Engineering and Computer Science, Faculty of Engineering, Kinki University, Hiroshima, Japan
Venue:
EuroPVM/MPI'06 Proceedings of the 13th European PVM/MPI User's Group conference on Recent advances in parallel virtual machine and message passing interface
Year:
2006

Citing 9
Cited 0

A high-performance, portable implementation of the MPI message passing interface standard

Parallel Computing
On implementing MPI-IO portably and with high performance

Proceedings of the sixth workshop on I/O in parallel and distributed systems
Optimizing noncontiguous accesses in MPI – IO

Parallel Computing
An Architecture of Stampi: MPI Library on a Cluster of Parallel Computers

Proceedings of the 7th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Stampi-I/O: A Flexible Parallel-I/O Library for Heterogeneous Computing Environment

Proceedings of the 9th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
An Abstract-Device Interface for Implementing Portable Parallel-I/O Interfaces

FRONTIERS '96 Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation
Parallel netCDF: A High-Performance Scientific I/O Interface

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Fast Parallel Non-Contiguous File Access

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
RFS: efficient and flexible remote file access for MPI-IO

CLUSTER '04 Proceedings of the 2004 IEEE International Conference on Cluster Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Parallel computation outputs intermediate data periodically, and typically the outputs are accessed for visualization in remote operation. To realize this kind of operations with derived data types among computers which have different MPI libraries, a Stampi library was proposed. For effective data-intensive I/O, a PVFS2 file system has been supported in its remote MPI-I/O operations by introducing MPICH as an underlying MPI library. This mechanism has been evaluated on interconnected PC clusters, and sufficient performance has been achieved with huge amount of data. In this paper, architecture, execution mechanism, and preliminary performance results are reported and discussed.