Effective nonblocking MPI-I/O in remote i/o operations using a multithreaded mechanism

Authors:
Yuichi Tsujita
Affiliations:
Department of Electronic Engineering and Computer Science, Faculty of Engineering, Kinki University, Hiroshima, Japan
Venue:
ISPA'04 Proceedings of the Second international conference on Parallel and Distributed Processing and Applications
Year:
2004

Citing 6
Cited 1

A high-performance, portable implementation of the MPI message passing interface standard

Parallel Computing
On implementing MPI-IO portably and with high performance

Proceedings of the sixth workshop on I/O in parallel and distributed systems
Improving Collective I/O Performance Using Threads

IPPS '99/SPDP '99 Proceedings of the 13th International Symposium on Parallel Processing and the 10th Symposium on Parallel and Distributed Processing
An Architecture of Stampi: MPI Library on a Cluster of Parallel Computers

Proceedings of the 7th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Stampi-I/O: A Flexible Parallel-I/O Library for Heterogeneous Computing Environment

Proceedings of the 9th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Evaluation of MPI Implementations on Grid-connected Clusters using an Emulated WAN Environment

CCGRID '03 Proceedings of the 3st International Symposium on Cluster Computing and the Grid

Improving web cache server performance through arbitral thread and delayed caching

Cluster Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

A flexible intermediate library named Stampi realizes seamless MPI operations on interconnected parallel computers. Dynamic process creation and MPI-I/O operations both inside a computer and among computers are available with it. MPI-I/O operations to a remote computer are realized by MPI-I/O processes of the Stampi library which are invoked on a remote computer using a vendor-supplied MPI-I/O library. If the vendor-supplied one is not available, a single MPI-I/O process is invoked on a remote computer, and it uses UNIX I/O functions instead of the vendor-supplied one. In nonblocking MPI-I/O functions with multiple user processes, the single MPI-I/O process carries out I/O operations required by the processes sequentially. This results in small overlap of computation by the user processes with I/O operations by the MPI-I/O process. Therefore performance of the nonblocking functions is poor with multiple user processes. To realize effective I/O operations, a Pthreads library has been implemented in the MPI-I/O mechanism, and multithreaded I/O operations have been realized. The newly implemented MPI-I/O mechanism has been evaluated on inter-connected PC clusters, and higher overlap of the computation with the I/O operations has been achieved.