STOC '97 Proceedings of the twenty-ninth annual ACM symposium on Theory of computing
Comparing random data allocation and data striping in multimedia servers
Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Efficient, distributed data placement strategies for storage area networks (extended abstract)
Proceedings of the twelfth annual ACM symposium on Parallel algorithms and architectures
GPFS: A Shared-Disk File System for Large Computing Clusters
FAST '02 Proceedings of the Conference on File and Storage Technologies
Hippodrome: Running Circles Around Storage Administration
FAST '02 Proceedings of the Conference on File and Storage Technologies
zFS " A Scalable Distributed File System Using Object Disks
MSS '03 Proceedings of the 20 th IEEE/11 th NASA Goddard Conference on Mass Storage Systems and Technologies (MSS'03)
MSS '03 Proceedings of the 20 th IEEE/11 th NASA Goddard Conference on Mass Storage Systems and Technologies (MSS'03)
SCADDAR: An Efficient Randomized Technique to Reorganize Continuous Media Blocks
ICDE '02 Proceedings of the 18th International Conference on Data Engineering
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
D-SPTF: decentralized request distribution in brick-based storage systems
ASPLOS XI Proceedings of the 11th international conference on Architectural support for programming languages and operating systems
FAB: building distributed enterprise disk arrays from commodity components
ASPLOS XI Proceedings of the 11th international conference on Architectural support for programming languages and operating systems
Evaluation of Distributed Recovery in Large-Scale Storage Systems
HPDC '04 Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing
A Self-Organizing Storage Cluster for Parallel Data-Intensive Applications
Proceedings of the 2004 ACM/IEEE conference on Supercomputing
The Panasas ActiveScale Storage Cluster: Delivering Scalable High Bandwidth Storage
Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Ceph: a scalable, high-performance distributed file system
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
Scalable security for large, high performance storage systems
Proceedings of the second ACM workshop on Storage security and survivability
Ceph: a scalable, high-performance distributed file system
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
Scalable security for petascale parallel file systems
Proceedings of the 2007 ACM/IEEE conference on Supercomputing
RADOS: a scalable, reliable storage service for petabyte-scale storage clusters
PDSW '07 Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07
PNUTS: Yahoo!'s hosted data serving platform
Proceedings of the VLDB Endowment
Proceedings of the 4th ACM international workshop on Storage security and survivability
Kinesis: A new approach to replica placement in distributed storage systems
ACM Transactions on Storage (TOS)
Redundant Data Placement Strategies for Cluster Storage Environments
OPODIS '08 Proceedings of the 12th International Conference on Principles of Distributed Systems
R-ADMAD: high reliability provision for large-scale de-duplication archival storage systems
Proceedings of the 23rd international conference on Supercomputing
A file assignment strategy independent of workload characteristic assumptions
ACM Transactions on Storage (TOS)
VisageFS dynamic storage features for wide-area workflows
PDCS '07 Proceedings of the 19th IASTED International Conference on Parallel and Distributed Computing and Systems
Chain replication in theory and in practice
Proceedings of the 9th ACM SIGPLAN workshop on Erlang
RSEDP: an effective hybrid data placement algorithm for large-scale storage systems
The Journal of Supercomputing
FastScale: accelerate RAID scaling by minimizing data migration
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
USENIXATC'11 Proceedings of the 2011 USENIX conference on USENIX annual technical conference
A load-aware data placement policy on cluster file system
NPC'11 Proceedings of the 8th IFIP international conference on Network and parallel computing
International Journal of Information and Communication Technology
Design and Evaluation of a New Approach to RAID-0 Scaling
ACM Transactions on Storage (TOS)
Performance and scalability evaluation of the Ceph parallel file system
PDSW '13 Proceedings of the 8th Parallel Data Storage Workshop
Scalable Metadata Management Through OSD+ Devices
International Journal of Parallel Programming
CRAID: online RAID upgrades using dynamic hot data reorganization
FAST'14 Proceedings of the 12th USENIX conference on File and Storage Technologies
| Hi-index | 0.00 |
Emerging large-scale distributed storage systems are faced with the task of distributing petabytes of data among tens or hundreds of thousands of storage devices. Such systems must evenly distribute data and workload to efficiently utilize available resources and maximize system performance, while facilitating system growth and managing hardware failures. We have developed CRUSH, a scalable pseudorandom data distribution function designed for distributed object-based storage systems that efficiently maps data objects to storage devices without relying on a central directory. Because large systems are inherently dynamic, CRUSH is designed to facilitate the addition and removal of storage while minimizing unnecessary data movement. The algorithm accommodates a wide variety of data replication and reliability mechanisms and distributes data in terms of user-defined policies that enforce separation of replicas across failure domains.