Tracking Files in the Kepler Provenance Framework

  • Authors:

  • Pierre Mouallem;Roselyne Barreto;Scott Klasky;Norbert Podhorszki;Mladen Vouk

  • Affiliations:

  • North Carolina State University, Raleigh NC 27695;Oak Ridge National Laboratory, Oak Ridge TN 37831;Oak Ridge National Laboratory, Oak Ridge TN 37831;Oak Ridge National Laboratory, Oak Ridge TN 37831;North Carolina State University, Raleigh NC 27695

  • Venue:
  • SSDBM 2009 Proceedings of the 21st International Conference on Scientific and Statistical Database Management
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Workflow Management Systems (WFMS), such as Kepler, are proving to be an important tool in scientific problem solving. They can automate and manage complex processes and huge amounts of data produced by petascale simulations. Typically, the produced data need to be properly visualized and analyzed by scientists in order to achieve the desired scientific goals. Both run-time and post analysis may benefit from, even require, additional meta-data --- provenance information. One of the challenges in this context is the tracking of the data files that can be produced in very large numbers during stages of the workflow, such as visualizations. The Kepler provenance framework collects all or part of the raw information flowing through the workflow graph. This information then needs to be further parsed to extract meta-data of interest. This can be done through add-on tools and algorithms. We show how to automate tracking specific information such as data files locations.