Approaches to Distributed Execution of Scientific Workflows in Kepler

  • Authors:

  • Marcin Płóciennik;Tomasz Żok;Ilkay Altintas;Jianwu Wang;Daniel Crawl;David Abramson;Frederic Imbeaux;Bernard Guillerminet;Marcos Lopez-Caniego;Isabel Campos Plasencia;Wojciech Pych;Pawel Ciecieląg;Bartek Palak;Michał Owsiak;Yann Frauel

  • Affiliations:

  • Poznań Supercomputing and Networking Center, IChB PAS, Poland. {marcinp,tzok}@man.poznan.pl;Poznań Supercomputing and Networking Center, IChB PAS, Poland. {marcinp,tzok}@man.poznan.pl;San Diego Supercomputer Center, University of California San Diego, USA. {altintas,jianwu,crawl}@sdsc.edu;San Diego Supercomputer Center, University of California San Diego, USA. {altintas,jianwu,crawl}@sdsc.edu;San Diego Supercomputer Center, University of California San Diego, USA. {altintas,jianwu,crawl}@sdsc.edu;Faculty of Information Technology, Monash University, Clayton, Australia. david.abramson@monash.edu;CEA, IRFM, France. {frederic.imbeaux,bernard.guillerminet}@cea.fr;CEA, IRFM, France. {frederic.imbeaux,bernard.guillerminet}@cea.fr;Instituto de Fisica de Cantabria, CSIC, Spain. {caniego,iscampos}@ifca.unican.es;Instituto de Fisica de Cantabria, CSIC, Spain. {caniego,iscampos}@ifca.unican.es;Nicolaus Copernicus Astronomical Center PAS, Poland. {pych,pci}@camk.edu.pl;Nicolaus Copernicus Astronomical Center PAS, Poland. {pych,pci}@camk.edu.pl;Poznań Supercomputing and Networking Center, IChB PAS, Poland. {bartek,michalo}@man.poznan.pl;Poznań Supercomputing and Networking Center, IChB PAS, Poland. {bartek,michalo}@man.poznan.pl;CEA, IRFM, France. yann.frauel@cea.fr

  • Venue:
  • Fundamenta Informaticae - Scalable Workflow Enactment Engines and Technology
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

The Kepler scientific workflow system enables creation, execution and sharing of workflows across a broad range of scientific and engineering disciplines while also facilitating remote and distributed execution of workflows. In this paper, we present and compare different approaches to distributed execution of workflows using the Kepler environment, including a distributed data-parallel framework using Hadoop and Stratosphere, and Cloud and Grid execution using Serpens, Nimrod/K and Globus actors. We also present real-life applications in computational chemistry, bioinformatics and computational physics to demonstrate the usage of different distributed computing capabilities of Kepler in executable workflows. We further analyze the differences of each approach and provide a guidance for their applications.