Adaptive Inference of Fine-grained Data Provenance to Achieve High Accuracy at Lower Storage Costs

Authors:
Mohammad Rezwanul Huq;Andreas Wombacher;Peter M. G. Apers
Affiliations:
-;-;-
Venue:
ESCIENCE '11 Proceedings of the 2011 IEEE Seventh International Conference on eScience
Year:
2011

Citing 0
Cited 2

Fine-grained provenance inference for a large processing chain with non-materialized intermediate views

SSDBM'12 Proceedings of the 24th international conference on Scientific and Statistical Database Management
Ariadne: managing fine-grained provenance on data streams

Proceedings of the 7th ACM international conference on Distributed event-based systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

In stream data processing, data arrives continuously and is processed by decision making, process control and e-science applications. To control and monitor these applications, reproducibility of result is a vital requirement. However, it requires massive amount of storage space to store fine-grained provenance data especially for those transformations with overlapping sliding windows. In this paper, we propose techniques which can significantly reduce storage costs and can achieve high accuracy. Our evaluation shows that adaptive inference technique can achieve almost 100% accurate provenance information for a given dataset at lower storage costs than the other techniques. Moreover, we present a guideline about the usage of different provenance collection techniques described in this paper based on the transformation operation and stream characteristics.