The HP AutoRAID hierarchical storage system
SOSP '95 Proceedings of the fifteenth ACM symposium on Operating systems principles
NFS illustrated
Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs
Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
OceanStore: an architecture for global-scale persistent storage
ASPLOS IX Proceedings of the ninth international conference on Architectural support for programming languages and operating systems
Chord: A scalable peer-to-peer lookup service for internet applications
Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications
Wide-area cooperative storage with CFS
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
A Toolkit for User-Level File Systems
Proceedings of the General Track: 2002 USENIX Annual Technical Conference
Managable Storage via Adaptation in WiND
CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Silverback: A Global-Scale Archival System
Silverback: A Global-Scale Archival System
Recovery Oriented Computing (ROC): Motivation, Definition, Techniques,
Recovery Oriented Computing (ROC): Motivation, Definition, Techniques,
Farsite: federated, available, and reliable storage for an incompletely trusted environment
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
Ivy: a read/write peer-to-peer file system
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
An analysis of internet content delivery systems
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
EW 10 Proceedings of the 10th workshop on ACM SIGOPS European workshop
High availability, scalable storage, dynamic peer networks: pick two
HOTOS'03 Proceedings of the 9th conference on Hot Topics in Operating Systems - Volume 9
Using random subsets to build scalable network services
USITS'03 Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems - Volume 4
AFRAID: a frequently redundant array of independent disks
ATEC '96 Proceedings of the 1996 annual conference on USENIX Annual Technical Conference
On the design of distributed protocols from differential equations
Proceedings of the twenty-third annual ACM symposium on Principles of distributed computing
Separating durability and availability in self-managed storage
Proceedings of the 11th workshop on ACM SIGOPS European workshop
Lessons and challenges in automating data dependability
Proceedings of the 11th workshop on ACM SIGOPS European workshop
Challenges in managing dependable data systems
ACM SIGMETRICS Performance Evaluation Review - Design, implementation, and performance of storage systems
On Composing Stream Applications in Peer-to-Peer Environments
IEEE Transactions on Parallel and Distributed Systems
Minimizing churn in distributed systems
Proceedings of the 2006 conference on Applications, technologies, architectures, and protocols for computer communications
On the road to recovery: restoring data after disasters
Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006
BitVault: a highly reliable distributed data retention platform
ACM SIGOPS Operating Systems Review - Systems work at Microsoft Research
Falling off the cliff: when systems go nonlinear
HOTOS'05 Proceedings of the 10th conference on Hot Topics in Operating Systems - Volume 10
Glacier: highly durable, decentralized storage despite massive correlated failures
NSDI'05 Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2
FUSE: lightweight guaranteed distributed failure notification
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
TFS: a transparent file system for contributory storage
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Antiquity: exploiting a secure log for wide-area distributed storage
Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007
Contributing storage using the transparent file system
ACM Transactions on Storage (TOS)
Enabling DVD-like features in P2P video-on-demand systems
Proceedings of the 2007 workshop on Peer-to-peer streaming and IP-TV
Insight into redundancy schemes in DHTs
The Journal of Supercomputing
Integration of domain-specific IT processes and tools in IBM service management
IBM Systems Journal
Stochastic analysis of the interplay between object maintenance and churn
Computer Communications
Replication degree customization for high availability
Proceedings of the 3rd ACM SIGOPS/EuroSys European Conference on Computer Systems 2008
RobuSTore: a distributed storage architecture with robust and high performance
Proceedings of the 2007 ACM/IEEE conference on Supercomputing
SafeStore: a durable and practical storage system
ATC'07 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference
Proactive replication in distributed storage systems using machine availability estimation
CoNEXT '07 Proceedings of the 2007 ACM CoNEXT conference
Near-optimal dynamic replication in unstructured peer-to-peer networks
Proceedings of the twenty-seventh ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
A new class of nature-inspired algorithms for self-adaptive peer-to-peer computing
ACM Transactions on Autonomous and Adaptive Systems (TAAS)
Friendstore: cooperative online backup using trusted nodes
Proceedings of the 1st Workshop on Social Network Systems
Improving peer-to-peer performance through server-side scheduling
ACM Transactions on Computer Systems (TOCS)
A framework for lazy replication in P2P VoD
Proceedings of the 18th International Workshop on Network and Operating Systems Support for Digital Audio and Video
A cost-effective distributed file service with QoS guarantees
Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware
AVMEM - availability-aware overlays for management operations in non-cooperative distributed systems
Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware
Dynamic nodeID based heterogeneity aware p2p system
Computer Communications
Reducing Replication Overhead for Data Durability in DHT Based P2P System
IEICE - Transactions on Information and Systems
Dynamic planning and weaving of dependability concerns for self-adaptive ubiquitous services
Proceedings of the 2009 ACM symposium on Applied Computing
DHT-Based Self-adapting Replication Protocol for Achieving High Data Availability
Advanced Internet Based Systems and Applications
Maintaining replicas in unstructured P2P systems
CoNEXT '08 Proceedings of the 2008 ACM CoNEXT Conference
Exploring data reliability tradeoffs in replicated storage systems
Proceedings of the 18th ACM international symposium on High performance distributed computing
EverLast: a distributed architecture for preserving the web
Proceedings of the 9th ACM/IEEE-CS joint conference on Digital libraries
Flexible, wide-area storage for distributed systems with WheelFS
NSDI'09 Proceedings of the 6th USENIX symposium on Networked systems design and implementation
Performance Analysis of Centralized versus Distributed Recovery Schemes in P2P Storage Systems
NETWORKING '09 Proceedings of the 8th International IFIP-TC 6 Networking Conference
Selfish Neighbor Selection in Peer-to-Peer Backup and Storage Applications
Euro-Par '09 Proceedings of the 15th International Euro-Par Conference on Parallel Processing
Finding Good Partners in Availability-Aware P2P Networks
SSS '09 Proceedings of the 11th International Symposium on Stabilization, Safety, and Security of Distributed Systems
Redundancy Maintenance and Garbage Collection Strategies in Peer-to-Peer Storage Systems
SSS '09 Proceedings of the 11th International Symposium on Stabilization, Safety, and Security of Distributed Systems
Self-organized Data Redundancy Management for Peer-to-Peer Storage Systems
IWSOS '09 Proceedings of the 4th IFIP TC 6 International Workshop on Self-Organizing Systems
Optimizing peer-to-peer backup using lifetime estimations
Proceedings of the 2009 EDBT/ICDT Workshops
Uncoordinated peer selection in P2P backup and storage applications
INFOCOM'09 Proceedings of the 28th IEEE international conference on Computer Communications Workshops
Cooperative recovery of distributed storage systems from multiple losses with network coding
IEEE Journal on Selected Areas in Communications
Performance analysis of peer-to-peer storage systems
ITC20'07 Proceedings of the 20th international teletraffic conference on Managing traffic performance in converged networks
Maintaining data reliability without availability in P2P storage systems
Proceedings of the 2010 ACM Symposium on Applied Computing
A cost-effective distributed file service with QoS guarantees
MIDDLEWARE2007 Proceedings of the 8th ACM/IFIP/USENIX international conference on Middleware
AVMEM: availability-aware overlays for management operations in non-cooperative distributed systems
MIDDLEWARE2007 Proceedings of the 8th ACM/IFIP/USENIX international conference on Middleware
Explicit construction of optimal exact regenerating codes for distributed storage
Allerton'09 Proceedings of the 47th annual Allerton conference on Communication, control, and computing
Tree-structured data regeneration in distributed storage systems with regenerating codes
INFOCOM'10 Proceedings of the 29th conference on Information communications
StrobeLight: lightweight availability mapping and anomaly detection
USENIX'09 Proceedings of the 2009 conference on USENIX Annual technical conference
Network coding for distributed storage systems
IEEE Transactions on Information Theory
Cost-bandwidth tradeoff in distributed storage systems
Computer Communications
Differentiated replication strategy in data centers
NPC'10 Proceedings of the 2010 IFIP international conference on Network and parallel computing
Data life time for different placement policies in P2P storage systems
Globe'10 Proceedings of the Third international conference on Data management in grid and peer-to-peer systems
A model for space-correlated failures in large-scale distributed systems
EuroPar'10 Proceedings of the 16th international Euro-Par conference on Parallel processing: Part I
Towards a more accurate availability evaluation in peer-to-peer storage systems
International Journal of High Performance Computing and Networking
A comparative study of rateless codes for P2P persistent torage
SSS'10 Proceedings of the 12th international conference on Stabilization, safety, and security of distributed systems
A quantitative analysis of redundancy schemes for peer-to- peer storage systems
SSS'10 Proceedings of the 12th international conference on Stabilization, safety, and security of distributed systems
Computer Networks: The International Journal of Computer and Telecommunications Networking
Securing P2P storage with a self-organizing payment scheme
DPM'10/SETOP'10 Proceedings of the 5th international Workshop on data privacy management, and 3rd international conference on Autonomous spontaneous security
Middleware for a re-configurable distributed archival store based on secret sharing
Proceedings of the ACM/IFIP/USENIX 11th International Conference on Middleware
Evaluation of p2p systems under different churn models: why we should bother
Euro-Par'11 Proceedings of the 17th international conference on Parallel processing - Volume Part I
Hybrid approaches for distributed storage systems
Globe'11 Proceedings of the 4th international conference on Data management in grid and peer-to-peer systems
Using emotional classification model for travel information system
International Journal of Computational Science and Engineering
Reducing data replication overhead in DHT based peer-to-peer system
HPCC'06 Proceedings of the Second international conference on High Performance Computing and Communications
Redundancy schemes for high availability in DHTs
ISPA'05 Proceedings of the Third international conference on Parallel and Distributed Processing and Applications
Understanding the session durability in peer-to-peer storage system
ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part IV
Long-term availability prediction for groups of volunteer resources
Journal of Parallel and Distributed Computing
Wayfinder: navigating and sharing information in a decentralized world
DBISP2P'04 Proceedings of the Second international conference on Databases, Information Systems, and Peer-to-Peer Computing
Heterogeneity aware p2p algorithm by using mobile nodeID
ICOIN'06 Proceedings of the 2006 international conference on Information Networking: advances in Data Communications and Wireless Networks
A novel data replication mechanism in P2P VoD system
Future Generation Computer Systems
Highly available DHTs: keeping data consistency after updates
AP2PC'05 Proceedings of the 4th international conference on Agents and Peer-to-Peer Computing
Contextual Trust Aided Enhancement of Data Availability in Peer-to-Peer Backup Storage Systems
Journal of Network and Systems Management
Choosing partners based on availability in P2P networks
ACM Transactions on Autonomous and Adaptive Systems (TAAS)
Efficient cooperative backup with decentralized trust management
ACM Transactions on Storage (TOS)
Redundantly grouped cross-object coding for repairable storage
Proceedings of the Asia-Pacific Workshop on Systems
Robust Redundancy Scheme for the Repair Process: Hierarchical Codes in the Bandwidth-Limited Systems
Journal of Grid Computing
Redundantly grouped cross-object coding for repairable storage
APSys'12 Proceedings of the Third ACM SIGOPS Asia-Pacific conference on Systems
Reducing Storage Overhead with Small Write Bottleneck Avoiding in Cloud RAID System
GRID '12 Proceedings of the 2012 ACM/IEEE 13th International Conference on Grid Computing
Reliability and availability issues in large-scale distributed systems
Proceedings of the Winter Simulation Conference
Peer-to-peer architectures for massively multiplayer online games: A Survey
ACM Computing Surveys (CSUR)
On the interplay between data redundancy and retrieval times in P2P storage systems
Computer Networks: The International Journal of Computer and Telecommunications Networking
FAST'14 Proceedings of the 12th USENIX conference on File and Storage Technologies
Hi-index | 0.06 |
Availability is a storage system property that is both highly desired and yet minimally engineered. While many systems provide mechanisms to improve availability - such as redundancy and failure recovery - how to best configure these mechanisms is typically left to the system manager. Unfortunately, few individuals have the skills to properly manage the trade-offs involved, let alone the time to adapt these decisions to changing conditions. Instead, most systems are configured statically and with only a cursory understanding of how the configuration will impact overall performance or availability. While this issue can be problematic even for individual storage arrays, it becomes increasingly important as systems are distributed - and absolutely critical for the wide-area peer-to-peer storage infrastructures being explored. This paper describes the motivation, architecture and implementation for a new peer-to-peer storage system, called TotalRecall, that automates the task of availability management. In particular, the TotalRecall system automatically measures and estimates the availability of its constituent host components, predicts their future availability based on past behavior, calculates the appropriate redundancy mechanisms and repair policies, and delivers user-specified availability while maximizing efficiency.