Automatic capture and efficient storage of e-Science experiment provenance

Authors:
Roger S. Barga;Luciano A. Digiampietri
Affiliations:
Microsoft Research, One Microsoft Way Redmond, WA 98052, U.S.A.;Institute of Computing, University of Campinas, Sao Paulo, Brazil
Venue:
Concurrency and Computation: Practice & Experience - The First Provenance Challenge
Year:
2008

Citing 0
Cited 18

Efficient provenance storage

Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Provenance and scientific workflows: challenges and opportunities

Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Provenance and the Price of Identity

Provenance and Annotation of Data and Processes
Efficient provenance storage over nested data collections

Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology
Managing Provenance in iRODS

ICCS 2009 Proceedings of the 9th International Conference on Computational Science
A navigation model for exploring scientific workflow provenance graphs

Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science
Techniques for efficiently querying scientific workflow provenance graphs

Proceedings of the 13th International Conference on Extending Database Technology
Fine-grained and efficient lineage querying of collection-based workflow provenance

Proceedings of the 13th International Conference on Extending Database Technology
RDFProv: A relational RDF store for querying and managing scientific workflow provenance

Data & Knowledge Engineering
Biocompute: towards a collaborative workspace for data intensive bio-science

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Towards a secure and efficient system for end-to-end provenance

TAPP'10 Proceedings of the 2nd conference on Theory and practice of provenance
The Foundations for Provenance on the Web

Foundations and Trends in Web Science
Data model for scientific models and hypotheses

The evolution of conceptual modeling
SciProv: an architecture for semantic query in provenance metadata on e-science context

ITBAM'11 Proceedings of the Second international conference on Information technology in bio- and medical informatics
Fine-grained provenance inference for a large processing chain with non-materialized intermediate views

SSDBM'12 Proceedings of the 24th international conference on Scientific and Statistical Database Management
A framework for scalable distributed provenance storage system

Computer Standards & Interfaces
Evaluation of a Hybrid Approach for Efficient Provenance Storage

ACM Transactions on Storage (TOS)
Automated data provenance capture in spreadsheets, with case studies

Future Generation Computer Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

For the first provenance challenge, we introduce a layered model to represent workflow provenance that allows navigation from an abstract model of the experiment to instance data collected during a specific experiment run. We outline modest extensions to a commercial workflow engine so it will automatically capture provenance at workflow runtime. We also present an approach to store this provenance data in a relational database. Finally, we demonstrate how core provenance queries in the challenge can be expressed in SQL and discuss the merits of our layered representation. Copyright © 2007 John Wiley & Sons, Ltd.