Scalable online comparative genomics of mononucleosomes: a BigJob

Authors:
Jack A. Smith;Melissa Romanus;Pradeep Kumar Mantha;Yaakoub El Khamra;Thomas C. Bishop;Shantenu Jha
Affiliations:
Marshall University, Huntington, WV;Rutgers University, Piscataway, NJ;Lawrence Berkeley Lab, Berkeley, CA;University of Texas - TACC, Austin, TX;Louisiana Tech University, Ruston, LA;Rutgers University, Piscataway, NJ
Venue:
Proceedings of the Conference on Extreme Science and Engineering Discovery Environment: Gateway to Discovery
Year:
2013

Citing 5
Cited 0

Kepler: An Extensible System for Design and Execution of Scientific Workflows

SSDBM '04 Proceedings of the 16th International Conference on Scientific and Statistical Database Management
Pegasus: A framework for mapping complex scientific workflows onto distributed systems

Scientific Programming
SAGA BigJob: An Extensible and Interoperable Pilot-Job Abstraction for Distributed Applications and Systems

CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
Running many molecular dynamics simulations on many supercomputers

Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging from the eXtreme to the campus and beyond
P∗: A model of pilot-abstractions

E-SCIENCE '12 Proceedings of the 2012 IEEE 8th International Conference on E-Science (e-Science)

Quantified Score

Hi-index	0.00

Visualization

Abstract

Our goal is to develop workflows for simulating arbitrary collections of mononucleosomes in atomic detail as an on demand analysis tool for online comparative genomics. The limiting factor is resource availability. The aim of this paper is to document and share our experiences in providing a general-purpose, easy-to-use and extensible solution for such computations. At the core it involves supporting the execution of high-throughput workloads of high-performance biomolecular simulations on one or more XSEDE machines. Although conceptually simple, it is still a difficult practical problem to solve, especially in a flexible, robust, scalable manner. Specifically, we employ BigJob-- an interoperable Pilot-Job. The bulk of this paper is about our experience in executing a very large number of ensembles including the associated non-trivial data management problem. Our experience suggests that although a nascent and fledgling technology, BigJob provides a flexible and scalable Pilot-Job to support workloads that were hitherto not easy.