Bridging HPC and grid file i/o with IOFSL

Authors:
Jason Cope;Kamil Iskra;Dries Kimpe;Robert Ross
Affiliations:
Mathematics and Computer Science Division, Argonne National Laboratory;Mathematics and Computer Science Division, Argonne National Laboratory;Computation Institute, University of Chicago / Argonne National Laboratory;Mathematics and Computer Science Division, Argonne National Laboratory
Venue:
PARA'10 Proceedings of the 10th international conference on Applied Parallel and Scientific Computing - Volume 2
Year:
2010

Citing 12
Cited 1

Supporting efficient execution in heterogeneous distributed computing environments with cactus and globus

Proceedings of the 2001 ACM/IEEE conference on Supercomputing
Grid Datafarm Architecture for Petascale Data Intensive Computing

CCGRID '02 Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid
MPICH-G2: a Grid-enabled implementation of the Message Passing Interface

Journal of Parallel and Distributed Computing - Special issue on computational grids
The Panasas ActiveScale Storage Cluster: Delivering Scalable High Bandwidth Storage

Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Parallel netCDF: A High-Performance Scientific I/O Interface

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Distributed computing in practice: the Condor experience: Research Articles

Concurrency and Computation: Practice & Experience - Grid Performance
A parallel I/O mechanism for distributed systems

CLUSTER '04 Proceedings of the 2004 IEEE International Conference on Cluster Computing
PVFS: a parallel file system for linux clusters

ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4
ZOID: I/O-forwarding infrastructure for petascale architectures

Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming
A new paradigm: Data-aware scheduling in grid computing

Future Generation Computer Systems
Optimization Techniques at the I/O Forwarding Layer

CLUSTER '10 Proceedings of the 2010 IEEE International Conference on Cluster Computing
GPFS: a shared-disk file system for large computing clusters

FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies

Optimizing I/O forwarding techniques for extreme-scale event tracing

Cluster Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Traditionally, little interaction has taken place between the Grid and high-performance computing (HPC) storage research communities. Grid research often focused on optimizing data accesses for high-latency, wide-area networks, while HPC research focused on optimizing data accesses for local, high-performance storage systems. Recent software and hardware trends are blurring the distinction between Grids and HPC. In this paper, we investigate the use of I/O forwarding -- a well established technique in leadership-class HPC machines-- in a Grid context. We show that the problems that triggered the introduction of I/O forwarding for HPC systems also apply to contemporary Grid computing environments. We present the design of our I/O forwarding infrastructure for Grid computing environments. Moreover, we discuss the advantages our infrastructure provides for Grids, such as simplified application data management in heterogeneous computing environments and support for multiple application I/O interfaces.