A Framework for Collecting Provenance in Data-Centric Scientific Workflows

  • Authors:

  • Yogesh L. Simmhan;Beth Plale;Dennis Gannon

  • Affiliations:

  • Indiana University, Bloomington IN 47405, USA;Indiana University, Bloomington IN 47405, USA;Indiana University, Bloomington IN 47405, USA

  • Venue:
  • ICWS '06 Proceedings of the IEEE International Conference on Web Services
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

The increasing ability for the earth sciences to sense the world around us is resulting in a growing need for datadriven applications that are under the control of data-centric workflows composed of grid- and web- services. The focus of our work is on provenance collection for these workflows, necessary to validate the workflow and to determine quality of generated data products. The challenge we address is to record uniform and usable provenance metadata that meets the domain needs while minimizing the modification burden on the service authors and the performance overhead on the workflow engine and the services. The framework, based on a loosely-coupled publish-subscribe architecture for propagating provenance activities, satisfies the needs of detailed provenance collection while a performance evaluation of a prototype finds a minimal performance overhead (in the range of 1% for an eight service workflow using 271 data products).