A hybrid approach for efficient provenance storage

Authors:
Yulai Xie;Dan Feng;Zhipeng Tan;Lei Chen;Kiran-Kumar Muniswamy-Reddy;Yan Li;Darrell D.E. Long
Affiliations:
Huazhong University of Science and Technology, Wuhan, China;Huazhong University of Science and Technology, Wuhan, China;Huazhong University of Science and Technology, Wuhan, China;Huazhong University of Science and Technology, Wuhan, China;Harvard University, Boston, MA, USA;University of California, Santa Cruz, Santa Cruz, CA, USA;University of California, Santa Cruz, Santa Cruz, CA, USA
Venue:
Proceedings of the 21st ACM international conference on Information and knowledge management
Year:
2012

Citing 6
Cited 4

The webgraph framework I: compression techniques

Proceedings of the 13th international conference on World Wide Web
A Framework for Collecting Provenance in Data-Centric Scientific Workflows

ICWS '06 Proceedings of the IEEE International Conference on Web Services
Provenance-aware storage systems

ATEC '06 Proceedings of the annual conference on USENIX '06 Annual Technical Conference
Recording and using provenance in a protein compressibility experiment

HPDC '05 Proceedings of the High Performance Distributed Computing, 2005. HPDC-14. Proceedings. 14th IEEE International Symposium
Efficient provenance storage

Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Provenance Information Model of Karma Version 3

SERVICES '09 Proceedings of the 2009 Congress on Services - I

HadoopProv: towards provenance as a first class citizen in MapReduce

TaPP'13 Proceedings of the 5th USENIX conference on Theory and Practice of Provenance
HadoopProv: towards provenance as a first class citizen in MapReduce

Proceedings of the 5th USENIX Workshop on the Theory and Practice of Provenance
Evaluation of a Hybrid Approach for Efficient Provenance Storage

ACM Transactions on Storage (TOS)
LogGC: garbage collecting audit log

Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security

Quantified Score

Hi-index	0.00

Visualization

Abstract

Efficient provenance storage is an essential step towards the adoption of provenance. In this paper, we analyze the provenance collected from multiple workloads with a view towards efficient storage. Based on our analysis, we characterize the properties of provenance with respect to long term storage. We then propose a hybrid scheme that takes advantage of the graph structure of provenance data and the inherent duplication in provenance data. Our evaluation indicates that our hybrid scheme, a combination of web graph compression (adapted for provenance) and dictionary encoding, provides the best tradeoff in terms of compression ratio, compression time and query performance when compared to other compression schemes.