Remote MPI-I/O on parallel virtual file system using a circular buffer for high throughput

  • Authors:
  • Y. Tsujita

  • Affiliations:
  • Kinki University, Takaya, Higashi-Hiroshima, Hiroshima, Japan

  • Venue:
  • International Journal of Computers and Applications
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

A flexible intermediate library named Stampi realizes seamless remote MPI-I/O operations on interconnected computers with the help of its MPI-I/O process which is invoked on a remote computer. The MPI-I/O process carries out I/O operations by using vendor's MPI-I/O library according to I/O requests from user processes. If the vendor's one is not available, UNIX I/O functions are used instead of it. A parallel virtual file system (PVFS) was supported in the remote MPI-I/O mechanism for data-intensive applications. Although this mechanism provided parallel I/O operations on a PVFS file system, its performance with UNIX I/O functions was low. Attempts to obtain high throughput have been made for this case with adopting a circular buffer mechanism in the MPI-I/O process to cache a part of or whole data. By optimizing configuration of the buffer, remote MPI-I/O operations with UNIX I/O functions outperformed those with direct calls of PVFS I/O functions on a PVFS file system.