Puffin: a fast and efficient web service-based condor job manager

  • Authors:
  • Jungha Woo;Jalaja Padma;Jae-Woo Lee;David Braun;Carol X. Song

  • Affiliations:
  • Purdue University, West Lafayette, IN;Purdue University, West Lafayette, IN;Purdue University, West Lafayette, IN;Purdue University, West Lafayette, IN;Purdue University, West Lafayette, IN

  • Venue:
  • Proceedings of the 5th Grid Computing Environments Workshop
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

For new Condor users, an easy-to-use interface to submit and monitor jobs from anywhere will significantly lower their barrier of entry to the Condor grid technology. Abstracting system and configuration details will facilitate more widespread use of Condor, as users with no detailed knowledge of Condor will be able to focus on their computational code. We introduce a context-free system that enables users to easily submit, manage, and monitor their job regardless of the operating system they use or the language by which the client is written. This system helps users to send computational jobs to a federated Condor resource, thus shielding users from details such as pool information, job migration, account used, etc..., so long as the users have valid credentials for job submission. Our system also features bulk job submission with load balancing among multiple schedulers, SpaceLease functionality to manage common data files, and asynchronous job event updates. Our system uses the JBoss application server and Condor Birdbath APIs. The design of this system aims at significantly reducing the overhead on the schedd which currently suffers from continuous polling from the users, and thus, helping Condor to scale in order to serve a very large number of concurrent users. A client application has been developed to test and validate various system components. Using this client application, a user can submit jobs from any operating system anywhere without logging in to the remote Condor submit machine, and is notified of the job status in real-time without any special configuration for the target environment.