A hypergraph partitioning based approach for scheduling of tasks with batch-shared I/O

Authors:
Gaurav Khanna;Nagavijayalakshmi Vydyanathan;T. Kurc;U. Catalyurek;P. Wyckoff;J. Saltz;P. Sadayappan
Affiliations:
The Ohio State University;The Ohio State University;The Ohio State University;The Ohio State University;Ohio Supercomputer Center;The Ohio State University;The Ohio State University
Venue:
CCGRID '05 Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2 - Volume 02
Year:
2005

Citing 0
Cited 14

Hypergraph partitioning for automatic memory hierarchy management

Proceedings of the 2006 ACM/IEEE conference on Supercomputing
Heuristics for scheduling file-sharing tasks on heterogeneous systems with distributed repositories

Journal of Parallel and Distributed Computing
An SCP-based heuristic approach for scheduling distributed data-intensive applications on global grids

Journal of Parallel and Distributed Computing
Multi-level direct K-way hypergraph partitioning with multiple constraints and fixed vertices

Journal of Parallel and Distributed Computing
Efficient reuse of replicated parallel data segments in computational grids

Future Generation Computer Systems
Using overlays for efficient data transfer over shared wide-area networks

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
A New Multi-level Algorithm Based on Particle Swarm Optimization for Bisecting Graph

ADMA '07 Proceedings of the 3rd international conference on Advanced Data Mining and Applications
A Set Coverage-based Mapping Heuristic for Scheduling Distributed Data-Intensive Applications on Global Grids

GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing
A data locality aware online scheduling approach for I/O-intensive jobs with file sharing

JSSPP'06 Proceedings of the 12th international conference on Job scheduling strategies for parallel processing
Scheduling of tasks with batch-shared I/O on heterogeneous systems

IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
Partitioning Hypergraphs in Scientific Computing Applications through Vertex Separators on Graphs

SIAM Journal on Scientific Computing
Scheduling file transfers for data-intensive jobs on heterogeneous clusters

Euro-Par'07 Proceedings of the 13th international Euro-Par conference on Parallel Processing
An effective refinement algorithm based on swarm intelligence for graph bipartitioning

ESCAPE'07 Proceedings of the First international conference on Combinatorics, Algorithms, Probabilistic and Experimental Methodologies
A new metric enabling an exact hypergraph model for the communication volume in distributed-memory parallel applications

Parallel Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper proposes a novel, hypergraph partitioning based strategy to schedule multiple data analysis tasks with batch-shared I/O behavior. This strategy formulates the sharing of files among tasks as a hypergraph to minimize the I/O overheads due to transferring of the same set of files multiple times and employs a dynamic scheme for file transfers to reduce contention on the storage system. We experimentally evaluate the proposed approach using application emulators from two application domains; analysis of remotely-sensed data and biomedical imaging.