A data locality aware online scheduling approach for I/O-intensive jobs with file sharing

Authors:
Gaurav Khanna;Umit Catalyurek;Tahsin Kurc;P. Sadayappan;Joel Saltz
Affiliations:
Dept. of Computer Science and Engineering, The Ohio State University;Dept. of Biomedical Informatics, The Ohio State University;Dept. of Biomedical Informatics, The Ohio State University;Dept. of Computer Science and Engineering, The Ohio State University;Dept. of Biomedical Informatics, The Ohio State University
Venue:
JSSPP'06 Proceedings of the 12th international conference on Job scheduling strategies for parallel processing
Year:
2006

Citing 14
Cited 2

Disk-directed I/O for MIMD multiprocessors

ACM Transactions on Computer Systems (TOCS)
Heuristics for Scheduling I/O Operations

IEEE Transactions on Parallel and Distributed Systems
Hypergraph-Partitioning-Based Decomposition for Parallel Sparse-Matrix Vector Multiplication

IEEE Transactions on Parallel and Distributed Systems
The AppLeS parameter sweep template: user-level middleware for the grid

Proceedings of the 2000 ACM/IEEE conference on Supercomputing
Using multicast to pre-load jobs on the ParPar cluster

Parallel Computing
Batch Scheduling in Parallel Database Systems

Proceedings of the Ninth International Conference on Data Engineering
Scheduling Multiple Data Visualization Query Workloads on a Shared Memory Machine

IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Dynamic Matching and Scheduling of a Class of Independent Tasks onto Heterogeneous Computing Systems

HCW '99 Proceedings of the Eighth Heterogeneous Computing Workshop
Heuristics for Scheduling Parameter Sweep Applications in Grid Environments

HCW '00 Proceedings of the 9th Heterogeneous Computing Workshop
Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications

HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
Pipeline and Batch Sharing in Grid Workloads

HPDC '03 Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing
The workload on parallel supercomputers: modeling the characteristics of rigid jobs

Journal of Parallel and Distributed Computing
An evaluation of the close-to-files processor and data co-allocation policy in multiclusters

CLUSTER '04 Proceedings of the 2004 IEEE International Conference on Cluster Computing
A hypergraph partitioning based approach for scheduling of tasks with batch-shared I/O

CCGRID '05 Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2 - Volume 02

Efficient reuse of replicated parallel data segments in computational grids

Future Generation Computer Systems
A probabilistic task scheduling method for grid environments

Future Generation Computer Systems

Quantified Score

Hi-index	0.01

Visualization

Abstract

Many scientific investigations have to deal with large amounts of data from simulations and experiments. Data analysis in such investigations typically involves extraction of subsets of data, followed by computations performed on extracted data. Scheduling in this context requires efficient utilization of the computational, storage and network resources to optimize response time. The data-intensive nature of such applications necessitates data-locality aware job scheduling algorithms. This paper proposes a hypergraph based dynamic scheduling heuristic for a stream of independent I/O intensive jobs with file sharing behavior. The proposed heuristic is based on an event-driven, run-time hypergraph modeling of the file sharing characteristics among jobs. Our experiments on a coupled compute/storage cluster show it performs better compared to previously proposed strategies, under a varying set of parameters for workloads from the application domain of biomedical image analysis.