Provenance management in Swift

Authors:
Luiz M. R. Gadelha Jr.;Ben Clifford;Marta Mattoso;Michael Wilde;Ian Foster
Affiliations:
Computer and Systems Engineering Program, Federal University of Rio de Janeiro, Brazil and National Laboratory for Scientific Computing, Brazil;Computation Institute, University of Chicago, USA;Computer and Systems Engineering Program, Federal University of Rio de Janeiro, Brazil;Computation Institute, University of Chicago, USA and Mathematics and Computer Science Division, Argonne National Laboratory, USA;Computation Institute, University of Chicago, USA and Mathematics and Computer Science Division, Argonne National Laboratory, USA
Venue:
Future Generation Computer Systems
Year:
2011

Citing 15
Cited 4

A worldwide flock of Condors: load sharing among workstation clusters

Future Generation Computer Systems - Special issue: resource management in distributed systems
Job Scheduling Under the Portable Batch System

IPPS '95 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
A Resource Management Architecture for Metacomputing Systems

IPPS/SPDP '98 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
Chimera: AVirtual Data System for Representing, Querying, and Automating Data Derivation

SSDBM '02 Proceedings of the 14th International Conference on Scientific and Statistical Database Management
A survey of data provenance in e-science

ACM SIGMOD Record
Tracking provenance in a virtual data grid

Concurrency and Computation: Practice & Experience - The First Provenance Challenge
Falkon: a Fast and Light-weight tasK executiON framework

Proceedings of the 2007 ACM/IEEE conference on Supercomputing
Workflows and e-Science: An overview of workflow system features and capabilities

Future Generation Computer Systems
Towards a Taxonomy of Provenance in Scientific Workflow Management Systems

SERVICES '09 Proceedings of the 2009 Congress on Services - I
Provenance Information Model of Karma Version 3

SERVICES '09 Proceedings of the 2009 Congress on Services - I
Parallel Scripting for Applications at the Petascale and Beyond

Computer
The Open Provenance Model core specification (v1.1)

Future Generation Computer Systems
XDTM: the XML data type and mapping for specifying datasets

EGC'05 Proceedings of the 2005 European conference on Advances in Grid Computing
Managing rapidly-evolving scientific workflows

IPAW'06 Proceedings of the 2006 international conference on Provenance and Annotation of Data
Applying the virtual data provenance model

IPAW'06 Proceedings of the 2006 international conference on Provenance and Annotation of Data

Exploring provenance in high performance scientific computing

Proceedings of the first annual workshop on High performance computing meets databases
MTCProv: a practical provenance query framework for many-task scientific computing

Distributed and Parallel Databases
Characterizing and profiling scientific workflows

Future Generation Computer Systems
Provenance traces of the swift parallel scripting system

Proceedings of the Joint EDBT/ICDT 2013 Workshops

Quantified Score

Hi-index	0.00

Visualization

Abstract

The Swift parallel scripting language allows for the specification, execution and analysis of large-scale computations in parallel and distributed environments. It incorporates a data model for recording and querying provenance information. In this article we describe these capabilities and evaluate the interoperability with other systems through the use of the Open Provenance Model. We describe Swift's provenance data model and compare it to the Open Provenance Model. We also describe and evaluate activities performed within the Third Provenance Challenge, which consisted of implementing a specific scientific workflow, capturing and recording provenance information of its execution, performing provenance queries, and exchanging provenance information with other systems. Finally, we propose improvements to both the Open Provenance Model and Swift's provenance system.