Provenance in collection-oriented scientific workflows

Authors:
Shawn Bowers;Timothy M. McPhillips;Bertram Ludäscher
Affiliations:
Genome Center, University of California, Davis, CA, U.S.A.;Genome Center, University of California, Davis, CA, U.S.A.;Genome Center, University of California, Davis, CA, U.S.A. and Department of Computer Science, University of California, Davis, CA, U.S.A.
Venue:
Concurrency and Computation: Practice & Experience - The First Provenance Challenge
Year:
2008

Citing 0
Cited 19

Provenance and scientific workflows: challenges and opportunities

Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Kepler/pPOD: Scientific Workflow and Provenance Support for Assembling the Tree of Life

Provenance and Annotation of Data and Processes
Advances and Challenges for Scalable Provenance in Stream Processing Systems

Provenance and Annotation of Data and Processes
Workflows and e-Science: An overview of workflow system features and capabilities

Future Generation Computer Systems
Scientific workflow design for mere mortals

Future Generation Computer Systems
Optimizing user views for workflows

Proceedings of the 12th International Conference on Database Theory
Detecting and resolving unsound workflow views for correct provenance analysis

Proceedings of the 2009 ACM SIGMOD International Conference on Management of data
Scientific Workflows: Business as Usual?

BPM '09 Proceedings of the 7th International Conference on Business Process Management
Fine-grained and efficient lineage querying of collection-based workflow provenance

Proceedings of the 13th International Conference on Extending Database Technology
Project histories: managing data provenance across collection-oriented scientific workflow runs

DILS'07 Proceedings of the 4th international conference on Data integration in the life sciences
A collaborative scheduling approach for service-driven scientific workflow execution

Journal of Computer and System Sciences
The Foundations for Provenance on the Web

Foundations and Trends in Web Science
Generating sound workflow views for correct provenance analysis

ACM Transactions on Database Systems (TODS)
Data integration systems for scientific applications

OTM'10 Proceedings of the 2010 international conference on On the move to meaningful internet systems
Putting lipstick on pig: enabling database-style workflow provenance

Proceedings of the VLDB Endowment
Hiding data and structure in workflow provenance

DNIS'11 Proceedings of the 7th international conference on Databases in Networked Information Systems
Provenance support for rework

TaPP'12 Proceedings of the 4th USENIX conference on Theory and Practice of Provenance
The providence of provenance

BNCOD'13 Proceedings of the 29th British National conference on Big Data
Towards semantic comparison of multi-granularity process traces

Knowledge-Based Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe a provenance model tailored to scientific workflows based on the collection-oriented modeling and design paradigm. Our implementation within the Kepler scientific workflow system captures the dependencies of data and collection creation events on preexisting data and collections, and embeds these provenance records within the data stream. A provenance query engine operates on self-contained workflow traces representing serializations of the output data stream for particular workflow runs. We demonstrate this approach in our response to the first provenance challenge. Copyright © 2007 John Wiley & Sons, Ltd.