Use of PVFS for Efficient Execution of Jobs with Pipeline-Shared I/O

Authors:
Nagavijayalakshmi Vydyanathan;Gaurav Khanna;Tahsin Kurc;Umit Catalyurek;Pete Wyckoff;Joel Saltz;P. Sadayappan
Affiliations:
The Ohio State University;The Ohio State University;The Ohio State University;The Ohio State University;Ohio Supercomputer Center;The Ohio State University;The Ohio State University
Venue:
GRID '04 Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing
Year:
2004

Citing 22
Cited 1

Application scheduling and processor allocation in multiprogrammed parallel processing systems

Performance Evaluation - Special issue: performance modeling of parallel processing systems
Robust partitioning policies of multiprocessor systems

Performance Evaluation - Special issue: performance modeling of parallel processing systems
Parallel application scheduling on networks of workstations

Journal of Parallel and Distributed Computing
Cluster I/O with River: making the fast case common

Proceedings of the sixth workshop on I/O in parallel and distributed systems
Static scheduling algorithms for allocating directed task graphs to multiprocessors

ACM Computing Surveys (CSUR)
The AppLeS parameter sweep template: user-level middleware for the grid

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
Distributed processing of very large datasets with DataCutter

Parallel Computing - Clusters and computational grids for scientific computing
A comparative study of online scheduling algorithms for networks of workstations

Cluster Computing
Using moldability to improve the performance of supercomputer jobs

Journal of Parallel and Distributed Computing
A Symbolic Approachto Modeling Cellular Behavior

HiPC '02 Proceedings of the 9th International Conference on High Performance Computing
DPFS: A Distributed Parallel File System

ICPP '02 Proceedings of the 2001 International Conference on Parallel Processing
Executing multiple pipelined data analysis operations in the grid

Proceedings of the 2002 ACM/IEEE conference on Supercomputing
Applying Chimera virtual data concepts to cluster finding in the Sloan Sky Survey

Proceedings of the 2002 ACM/IEEE conference on Supercomputing
Armada: A Parallel File System for Computational Grids

CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Mapping heterogeneous task graphs onto heterogeneous system graphs

HCW '97 Proceedings of the 6th Heterogeneous Computing Workshop (HCW '97)
dQUOB: Managing Large Data Flows Using Dynamic Embedded Queries

HPDC '00 Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing
Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications

HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
Pipeline and Batch Sharing in Grid Workloads

HPDC '03 Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing
Exploiting Functional Decomposition for Efficient Parallel Processing of Multiple Data Analysis Queries

IPDPS '03 Proceedings of the 17th International Symposium on Parallel and Distributed Processing
Condor-G: A Computation Management Agent for Multi-Institutional Grids

HPDC '01 Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing
IBM Storage Tank-- A heterogeneous scalable SAN file system

IBM Systems Journal
PVFS: a parallel file system for linux clusters

ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4

Dynamic load balancing for I/O-intensive applications on clusters

ACM Transactions on Storage (TOS)

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper is concerned with efficient execution of applications that are composed of chain of sequential data processes, which exchange data through a file system. We focus on pipeline-shared I/O behavior within a single pipeline of processes running on a cluster. We examine several scheduling strategies and experimentally evaluate them for efficient use the Parallel Virtual File System (PVFS) as a common storage pool.