Towards a common model for pilot-jobs

Authors:
Andre Luckow;Mark Santcroos;Ole Weidner;Andre Merzky;Sharath Maddineni;Shantenu Jha
Affiliations:
LSU, Baton Rouge, LA, USA;Bioinformatics Laboratory, Academic Medical Center University of Amsterdam,, Amsterdam, Netherlands;LSU, Baton Rouge, LA, USA;LSU, Baton Rouge, LA, USA;LSU, Baton Rouge, LA, USA;Rutgers University, Piscataway, NJ, USA
Venue:
Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing
Year:
2012

Citing 2
Cited 2

Condor-G: A Computation Management Agent for Multi-Institutional Grids

Cluster Computing
SAGA BigJob: An Extensible and Interoperable Pilot-Job Abstraction for Distributed Applications and Systems

CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing

Pilot-MapReduce: an extensible and flexible MapReduce implementation for distributed data

Proceedings of third international workshop on MapReduce and its Applications Date
A science-gateway workload archive to study pilot jobs, user activity, bag of tasks, task sub-steps, and workflow executions

Euro-Par'12 Proceedings of the 18th international conference on Parallel processing workshops

Quantified Score

Hi-index	0.00

Visualization

Abstract

Pilot-Jobs have become one of the most successful abstractions in distributed computing. In spite of extensive uptake, there does not exist a well defined, unifying conceptual model of pilot-jobs which can be used to define, compare and contrast different implementations. This presents a barrier to extensibility and interoperability. This paper is an attempt to, (i) provide a minimal but complete model (P*) of pilot-jobs, (ii) establish the generality of the P* Model by mapping various existing and well known pilot-jobs frameworks such as Condor and DIANE to P*, (iii) demonstrate the interoperable and concurrent usage of distinct pilot-job frameworks on different production distributed cyberinfrastructures via the use of an extensible API for the P* Model (Pilot-API).