Hint controlled distribution with parallel file systems

Authors:
Hipolito Vasquez Lucas;Thomas Ludwig
Affiliations:
Parallele und Verteilte Systeme, Institut für Informatik, Ruprecht-Karls-Universität Heidelberg, Heidelberg, Germany;Parallele und Verteilte Systeme, Institut für Informatik, Ruprecht-Karls-Universität Heidelberg, Heidelberg, Germany
Venue:
PVM/MPI'05 Proceedings of the 12th European PVM/MPI users' group conference on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Year:
2005

Citing 12
Cited 1

A case for redundant arrays of inexpensive disks (RAID)

SIGMOD '88 Proceedings of the 1988 ACM SIGMOD international conference on Management of data
Input/output behavior of supercomputing applications

Proceedings of the 1991 ACM/IEEE conference on Supercomputing
On implementing MPI-IO portably and with high performance

Proceedings of the sixth workshop on I/O in parallel and distributed systems
Maximizing performance in a striped disk array

ISCA '90 Proceedings of the 17th annual international symposium on Computer Architecture
A case for using MPI's derived datatypes to improve I/O performance

SC '98 Proceedings of the 1998 ACM/IEEE conference on Supercomputing
Exploiting global input/output access pattern classification

SC '97 Proceedings of the 1997 ACM/IEEE conference on Supercomputing
An overview of cluster computing

Beowulf cluster computing with Linux
Using MPI-2: Advanced Features of the Message Passing Interface

Using MPI-2: Advanced Features of the Message Passing Interface
Implementation and performance of a parallel file system for high performance distributed applications

HPDC '96 Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing
Automatic classification of input/output access patterns

Automatic classification of input/output access patterns
The performance impact of I/O optimizations and disk improvements

IBM Journal of Research and Development
Characteristics of I/O traffic in personal computer and server workloads

IBM Systems Journal

A New File-Specific Stripe Size Selection Method for Highly Concurrent Data Access

GRID '12 Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

The performance of scientific parallel programs with high file-I/O-activity running on top of cluster computers strongly depends on the qualitative and quantitative characteristics of the requested I/O-accesses. It also depends on the corresponding mechanisms and policies being used at the parallel file system level. This paper presents the motivation and design of a set of MPI-IO-hints. These hints are used to select the distribution function with which a parallel file system manipulates an opened file. The implementation of a new physical distribution function called varstrip_dist is also presented in this article. This function is proposed based upon spatial characteristics presented by I/O-access patterns observed at the application level.