Antiquity: exploiting a secure log for wide-area distributed storage
Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007
Gossiping over storage systems is practical
ACM SIGOPS Operating Systems Review - Gossip-based computer networking
Stochastic analysis of the interplay between object maintenance and churn
Computer Communications
Proactive replication in distributed storage systems using machine availability estimation
CoNEXT '07 Proceedings of the 2007 ACM CoNEXT conference
A data placement service for petascale applications
PDSW '07 Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07
Durability of replicated distributed storage systems
SIGMETRICS '08 Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Near-optimal dynamic replication in unstructured peer-to-peer networks
Proceedings of the twenty-seventh ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
StoreGPU: exploiting graphics processing units to accelerate distributed storage systems
HPDC '08 Proceedings of the 17th international symposium on High performance distributed computing
UsenetDHT: a low-overhead design for Usenet
NSDI'08 Proceedings of the 5th USENIX Symposium on Networked Systems Design and Implementation
Friendstore: cooperative online backup using trusted nodes
Proceedings of the 1st Workshop on Social Network Systems
Improving peer-to-peer performance through server-side scheduling
ACM Transactions on Computer Systems (TOCS)
Data placement for scientific applications in distributed environments
GRID '07 Proceedings of the 8th IEEE/ACM International Conference on Grid Computing
AVMEM - availability-aware overlays for management operations in non-cooperative distributed systems
Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware
Tiered fault tolerance for long-term integrity
FAST '09 Proccedings of the 7th conference on File and storage technologies
Analysis of user-driven peer selection in peer-to-peer backup and storage systems
Proceedings of the 3rd International Conference on Performance Evaluation Methodologies and Tools
Exploring data reliability tradeoffs in replicated storage systems
Proceedings of the 18th ACM international symposium on High performance distributed computing
EverLast: a distributed architecture for preserving the web
Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries
On GPU's viability as a middleware accelerator
Cluster Computing
Selfish Neighbor Selection in Peer-to-Peer Backup and Storage Applications
Euro-Par '09 Proceedings of the 15th International Euro-Par Conference on Parallel Processing
Finding Good Partners in Availability-Aware P2P Networks
SSS '09 Proceedings of the 11th International Symposium on Stabilization, Safety, and Security of Distributed Systems
Maintaining data reliability without availability in P2P storage systems
Proceedings of the 2010 ACM Symposium on Applied Computing
An analytical estimation of durability in DHTs
ICDCIT'07 Proceedings of the 4th international conference on Distributed computing and internet technology
AVMEM: availability-aware overlays for management operations in non-cooperative distributed systems
MIDDLEWARE2007 Proceedings of the 8th ACM/IFIP/USENIX international conference on Middleware
TrustCode: P2P reputation-based trust management using network coding
HiPC'08 Proceedings of the 15th international conference on High performance computing
Lithium: virtual machine storage for the cloud
Proceedings of the 1st ACM symposium on Cloud computing
RACS: a case for cloud storage diversity
Proceedings of the 1st ACM symposium on Cloud computing
MOON: MapReduce On Opportunistic eNvironments
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
A GPU accelerated storage system
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Object storage on CRAQ: high-throughput chain replication for read-mostly workloads
USENIX'09 Proceedings of the 2009 conference on USENIX Annual technical conference
Network coding for distributed storage systems
IEEE Transactions on Information Theory
AmazingStore: available, low-cost online storage service using cloudlets
IPTPS'10 Proceedings of the 9th international conference on Peer-to-peer systems
Differentiated replication strategy in data centers
NPC'10 Proceedings of the 2010 IFIP international conference on Network and parallel computing
Data life time for different placement policies in P2P storage systems
Globe'10 Proceedings of the Third international conference on Data management in grid and peer-to-peer systems
Availability in globally distributed storage systems
OSDI'10 Proceedings of the 9th USENIX conference on Operating systems design and implementation
A quantitative analysis of redundancy schemes for peer-to- peer storage systems
SSS'10 Proceedings of the 12th international conference on Stabilization, safety, and security of distributed systems
Small trusted primitives for dependable systems
ACM SIGOPS Operating Systems Review
Computer Networks: The International Journal of Computer and Telecommunications Networking
Ensuring content integrity for untrusted peer-to-peer content distribution networks
NSDI'07 Proceedings of the 4th USENIX conference on Networked systems design & implementation
A novel and optimal video replication technique for video-on-demand systems
Proceedings of the International Conference & Workshop on Emerging Trends in Technology
Replica placement in peer-assisted clouds: an economic approach
Proceedings of the 11th IFIP WG 6.1 international conference on Distributed applications and interoperable systems
Reducing Repair Traffic in P2P Backup Systems: Exact Regenerating Codes on Hierarchical Codes
ACM Transactions on Storage (TOS)
Hybrid approaches for distributed storage systems
Globe'11 Proceedings of the 4th international conference on Data management in grid and peer-to-peer systems
Fast Access to Distributed Atomic Memory
SIAM Journal on Computing
Contextual Trust Aided Enhancement of Data Availability in Peer-to-Peer Backup Storage Systems
Journal of Network and Systems Management
P2P and cloud: a marriage of convenience for replica management
IWSOS'12 Proceedings of the 6th IFIP TC 6 international conference on Self-Organizing Systems
Choosing partners based on availability in P2P networks
ACM Transactions on Autonomous and Adaptive Systems (TAAS)
Multimedia Tools and Applications
Reliable MapReduce computing on opportunistic resources
Cluster Computing
Efficient cooperative backup with decentralized trust management
ACM Transactions on Storage (TOS)
Erasure coding in windows azure storage
USENIX ATC'12 Proceedings of the 2012 USENIX conference on Annual Technical Conference
Robust Redundancy Scheme for the Repair Process: Hierarchical Codes in the Bandwidth-Limited Systems
Journal of Grid Computing
High throughput computing over peer-to-peer networks
Future Generation Computer Systems
Scalia: an adaptive scheme for efficient multi-cloud storage
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Peer-to-peer architectures for massively multiplayer online games: A Survey
ACM Computing Surveys (CSUR)
Durable data storage in distributed non persistent caching environment
Proceedings of the 6th ACM India Computing Convention
Copysets: reducing the frequency of data loss in cloud storage
USENIX ATC'13 Proceedings of the 2013 USENIX conference on Annual Technical Conference
On the interplay between data redundancy and retrieval times in P2P storage systems
Computer Networks: The International Journal of Computer and Telecommunications Networking
Hi-index | 0.06 |
This paper considers replication strategies for storage systems that aggregate the disks of many nodes spread over the Internet. Maintaining replication in such systems can be prohibitively expensive, since every transient network or host failure could potentially lead to copying a server's worth of data over the Internet to maintain replication levels. The following insights in designing an efficient replication algorithm emerge from the paper's analysis. First, durability can be provided separately from availability; the former is less expensive to ensure and a more useful goal for many wide-area applications. Second, the focus of a durability algorithm must be to create new copies of data objects faster than permanent disk failures destroy the objects; careful choice of policies for what nodes should hold what data can decrease repair time. Third, increasing the number of replicas of each data object does not help a system tolerate a higher disk failure probability, but does help tolerate bursts of failures. Finally, ensuring that the system makes use of replicas that recover after temporary failure is critical to efficiency. Based on these insights, the paper proposes the Carbonite replication algorithm for keeping data durable at a low cost. A simulation of Carbonite storing 1 TB of data over a 365 day trace of PlanetLab activity shows that Carbonite is able to keep all data durable and uses 44% more network traffic than a hypothetical system that only responds to permanent failures. In comparison, Total Recall and DHash require almost a factor of two more network traffic than this hypothetical system.