Conflict Detection Algorithm to Minimize Locking for MPI-IO Atomicity

  • Authors:

  • Saba Sehrish;Jun Wang;Rajeev Thakur

  • Affiliations:

  • School of Electrical Engineering and Computer Science, University of Central Florida, Orlando, USA 32816;School of Electrical Engineering and Computer Science, University of Central Florida, Orlando, USA 32816;Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, USA 60439

  • Venue:
  • Proceedings of the 16th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Many scientific applications require high-performance concurrent I/O accesses to a file by multiple processes. Those applications rely indirectly on atomic I/O capabilities in order to perform updates to structured datasets, such as those stored in HDF5 format files. Current support for atomicity in MPI-IO is provided by locking around the operations, imposing lock overhead in all situations, even though in many cases these operations are non-overlapping in the file. We propose to isolate non-overlapping accesses from overlapping ones in independent I/O cases, allowing the non-overlapping ones to proceed without imposing lock overhead. To enable this, we have implemented an efficient conflict detection algorithm in MPI-IO using MPI file views and datatypes. We show that our conflict detection scheme incurs minimal overhead on I/O operations, making it an effective mechanism for avoiding locks when they are not needed.