Inferring fine-grained data provenance in stream data processing: reduced storage cost, high accuracy

Authors:
Mohammad Rezwanul Huq;Andreas Wombacher;Peter M. G. Apers
Affiliations:
University of Twente, The Netherlands;University of Twente, The Netherlands;University of Twente, The Netherlands
Venue:
DEXA'11 Proceedings of the 22nd international conference on Database and expert systems applications - Volume Part II
Year:
2011

Citing 9
Cited 3

Models and issues in data stream systems

Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Lineage tracing for general data warehouse transformations

The VLDB Journal — The International Journal on Very Large Data Bases
Aurora: a new model and architecture for data stream management

The VLDB Journal — The International Journal on Very Large Data Bases
A survey of data provenance in e-science

ACM SIGMOD Record
Provenance in databases

Proceedings of the 2007 ACM SIGMOD international conference on Management of data
Efficient provenance storage

Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Provenance in Sensornet Republishing

Provenance and Annotation of Data and Processes
Facilitating fine grained data provenance using temporal data model

Proceedings of the Seventh International Workshop on Data Management for Sensor Networks
LIVE: a lineage-supported versioned DBMS

SSDBM'10 Proceedings of the 22nd international conference on Scientific and statistical database management

Fine-grained provenance inference for a large processing chain with non-materialized intermediate views

SSDBM'12 Proceedings of the 24th international conference on Scientific and Statistical Database Management
Reconstructing provenance

ISWC'12 Proceedings of the 11th international conference on The Semantic Web - Volume Part II
ProvenanceCurious: a tool to infer data provenance from scripts

Proceedings of the 16th International Conference on Extending Database Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

Fine-grained data provenance ensures reproducibility of results in decision making, process control and e-science applications. However, maintaining this provenance is challenging in stream data processing because of its massive storage consumption, especially with large overlapping sliding windows. In this paper, we propose an approach to infer fine-grained data provenance by using a temporal data model and coarse-grained data provenance of the processing. The approach has been evaluated on a real dataset and the result shows that our proposed inferring method provides provenance information as accurate as explicit fine-grained provenance at reduced storage consumption.