Using Semantic Information to Guide Efficient Parallel I/O on Clusters

  • Authors:
  • Martin Schulz

  • Affiliations:
  • -

  • Venue:
  • HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

Despite the large I/O capabilities in modern cluster architectures with local disks on each node, applications mostly are not enabled to fully exploit them. This is especially problematic for data intensive applications which often suffer from low I/O performance. As one solution for this problem, a Distribution I/O Management (DIOM) system has been developed to manage a transparent distribution of data across cluster nodes and to then allow applicationsto access this data purely from local disks. In order to be effective, however, this distribution process requires semantic information about both the application and the input data. This work therefore extends DIOM to include independent specifications for both data formats and application I/O patterns and thereby decouples them. This work is driven by an application from nuclear medical imaging, the reconstruction of PET images, for which DIOM has provento be an adequate solution enabling truly scalable I/O and thereby improving the overall application performance.