Redundancy schemes for high availability in DHTs

Authors:
Fan Wu;Tongqing Qiu;Yuequan Chen;Guihai Chen
Affiliations:
State Key Laboratory of Novel Software Technology, Nanjing University, China;State Key Laboratory of Novel Software Technology, Nanjing University, China;State Key Laboratory of Novel Software Technology, Nanjing University, China;State Key Laboratory of Novel Software Technology, Nanjing University, China
Venue:
ISPA'05 Proceedings of the Third international conference on Parallel and Distributed Processing and Applications
Year:
2005

Citing 15
Cited 2

Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web

STOC '97 Proceedings of the twenty-ninth annual ACM symposium on Theory of computing
A digital fountain approach to reliable distribution of bulk data

Proceedings of the ACM SIGCOMM '98 conference on Applications, technologies, architectures, and protocols for computer communication
Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs

Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
OceanStore: an architecture for global-scale persistent storage

ASPLOS IX Proceedings of the ninth international conference on Architectural support for programming languages and operating systems
Chord: A scalable peer-to-peer lookup service for internet applications

Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
A scalable content-addressable network

Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility

SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Wide-area cooperative storage with CFS

SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Erasure Coding Vs. Replication: A Quantitative Comparison

IPTPS '01 Revised Papers from the First International Workshop on Peer-to-Peer Systems
Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems

Middleware '01 Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg
Measuring End-User Availability on the Web: Practical Experience

DSN '02 Proceedings of the 2002 International Conference on Dependable Systems and Networks
High availability, scalable storage, dynamic peer networks: pick two

HOTOS'03 Proceedings of the 9th conference on Hot Topics in Operating Systems - Volume 9
Designing a DHT for low latency and high throughput

NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
Total recall: system support for automated availability management

NSDI'04 Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation - Volume 1
High availability in DHTs: erasure coding vs. replication

IPTPS'05 Proceedings of the 4th international conference on Peer-to-Peer Systems

Towards the design of optimal data redundancy schemes for heterogeneous cloud storage infrastructures

Computer Networks: The International Journal of Computer and Telecommunications Networking
SEERDIS: a DHT-based resource indexing and discovery scheme for the data center

Proceedings of the 19th High Performance Computing Symposia

Quantified Score

Hi-index	0.00

Visualization

Abstract

High availability in peer-to-peer DHTs requires data redundancy. This paper takes user download behavior into account to evaluate redundancy schemes in data storage and share systems. Furthermore, we propose a hybrid redundancy scheme of replication and erasure coding. Experiment results show that replication scheme saves more bandwidth than erasure coding scheme, although it requires more storage space, when average node availability is higher than 48%. Our hybrid scheme saves more maintenance bandwidth with acceptable redundancy factor.