Providing High Reliability in a Minimum Redundancy Archival Storage System

Authors:
Deepavali Bhagwat;Kristal Pollack;Darrell D. E. Long;Thomas Schwarz;Ethan L. Miller;Jehan-Francois Paris
Affiliations:
University of California, Santa Cruz, USA;University of California, Santa Cruz, USA;University of California, Santa Cruz, USA;University of California, Santa Cruz, USA;University of California, Santa Cruz, USA;University of Houston, USA
Venue:
MASCOTS '06 Proceedings of the 14th IEEE International Symposium on Modeling, Analysis, and Simulation
Year:
2006

Citing 0
Cited 11

Evaluating the usefulness of content addressable storage for high-performance data intensive applications

HPDC '08 Proceedings of the 17th international symposium on High performance distributed computing
Secure data deduplication

Proceedings of the 4th ACM international workshop on Storage security and survivability
R-ADMAD: high reliability provision for large-scale de-duplication archival storage systems

Proceedings of the 23rd international conference on Supercomputing
Self-adaptive disk arrays

SSS'06 Proceedings of the 8th international conference on Stabilization, safety, and security of distributed systems
Reliability analysis of deduplicated and erasure-coded storage

ACM SIGMETRICS Performance Evaluation Review
PRESIDIO: A Framework for Efficient Archival Data Storage

ACM Transactions on Storage (TOS)
Anchor-driven subchunk deduplication

Proceedings of the 4th Annual International Conference on Systems and Storage
SeWDReSS: on the design of an application independent, secure, wide-area disaster recovery storage system

Multimedia Tools and Applications
Understanding data survivability in archival storage systems

Proceedings of the 5th Annual International Systems and Storage Conference
Read-Performance Optimization for Deduplication-Based Storage Systems in the Cloud

ACM Transactions on Storage (TOS)
SBBS: A sliding blocking algorithm with backtracking sub-blocks for duplicate data detection

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Inter-file compression techniques store files as sets of references to data objects or chunks that can be shared among many files. While these techniques can achieve much better compression ratios than conventional intra-file compression methods such as Lempel-Ziv compression, they also reduce the reliability of the storage system because the loss of a few critical chunks can lead to the loss of many files. We show how to eliminate this problem by choosing for each chunk a replication level that is a function of the amount of data that would be lost if that chunk were lost. Experiments using actual archival data show that our technique can achieve significantly higher robustness than a conventional approach combining data mirroring and intra-file compression while requiring about half the storage space.