A case for redundant arrays of inexpensive disks (RAID)
SIGMOD '88 Proceedings of the 1988 ACM SIGMOD international conference on Management of data
Beating the I/O bottleneck: a case for log-structured file systems
ACM SIGOPS Operating Systems Review
IBM Systems Journal
IBM Systems Journal
Comparison of sparing alternatives for disk arrays
ISCA '92 Proceedings of the 19th annual international symposium on Computer architecture
On-line data compression in a log-structured file system
ASPLOS V Proceedings of the fifth international conference on Architectural support for programming languages and operating systems
Non-volatile memory for fast, reliable file systems
ASPLOS V Proceedings of the fifth international conference on Architectural support for programming languages and operating systems
Parity declustering for continuous operation in redundant disk arrays
ASPLOS V Proceedings of the fifth international conference on Architectural support for programming languages and operating systems
The design and implementation of a log-structured file system
ACM Transactions on Computer Systems (TOCS)
The architecture of a fault-tolerant cached RAID controller
ISCA '93 Proceedings of the 20th annual international symposium on computer architecture
The logical disk: a new approach to improving file systems
SOSP '93 Proceedings of the fourteenth ACM symposium on Operating systems principles
Adaptive block rearrangement
Tcl and the Tk toolkit
Background data movement in a log-structured disk subsystem
IBM Journal of Research and Development
RAID: high-performance, reliable secondary storage
ACM Computing Surveys (CSUR)
The TickerTAIP parallel RAID architecture
ACM Transactions on Computer Systems (TOCS)
Performance '93 Proceedings of the 16th IFIP Working Group 7.3 international symposium on Computer performance modeling measurement and evaluation
Minimizing mean seek distance in mirrored disk systems by cylinder remapping
Performance '93 Proceedings of the 16th IFIP Working Group 7.3 international symposium on Computer performance modeling measurement and evaluation
Dynamic parity stripe reorganizations for RAID5 disk arrays
PDIS '94 Proceedings of the third international conference on on Parallel and distributed information systems
Directory Reference Patterns in Hierarchical File Systems
IEEE Transactions on Knowledge and Data Engineering
File Migration on the Cray Y-MP at the National Center for Atmospheric
File Migration on the Cray Y-MP at the National Center for Atmospheric
Robo-line Storage: Low Latency, High Capacity Storage Systems over
Robo-line Storage: Low Latency, High Capacity Storage Systems over
TCON'95 Proceedings of the USENIX 1995 Technical Conference Proceedings
File system logging versus clustering: a performance comparison
TCON'95 Proceedings of the USENIX 1995 Technical Conference Proceedings
Heuristic cleaning algorithms in log-structured file systems
TCON'95 Proceedings of the USENIX 1995 Technical Conference Proceedings
Strategic directions in storage I/O issues in large-scale computing
ACM Computing Surveys (CSUR) - Special ACM 50th-anniversary issue: strategic directions in computing research
Improving the performance of log-structured file systems with adaptive methods
Proceedings of the sixteenth ACM symposium on Operating systems principles
Capacity planning with phased workloads
Proceedings of the 1st international workshop on Software and performance
Phoenix: a low-power fault-tolerant real-time network-attached storage device
MULTIMEDIA '99 Proceedings of the seventh ACM international conference on Multimedia (Part 1)
OceanStore: an architecture for global-scale persistent storage
ASPLOS IX Proceedings of the ninth international conference on Architectural support for programming languages and operating systems
ACM Transactions on Database Systems (TODS)
Reliability and performance of hierarchical RAID with multiple controllers
Proceedings of the twentieth annual ACM symposium on Principles of distributed computing
Orthogonal Striping and Mirroring in Distributed RAID for I/O-Centric Cluster Computing
IEEE Transactions on Parallel and Distributed Systems
Experiences with VI communication for database storage
ISCA '02 Proceedings of the 29th annual international symposium on Computer architecture
RAPID-Cache-A Reliable and Inexpensive Write Cache for High Performance Storage Systems
IEEE Transactions on Parallel and Distributed Systems
Models of Parallel Applications with Large Computation and I/O Requirements
IEEE Transactions on Software Engineering
LegionFS: a secure and scalable file system supporting cross-domain high-performance applications
Proceedings of the 2001 ACM/IEEE conference on Supercomputing
Cluster Computing
On using network RAM as a non-volatile buffer
Cluster Computing
A Gracefully Degradable Declustered RAID Architecture
Cluster Computing
LH*G: A High-Availability Scalable Distributed Data Structure By Record Grouping
IEEE Transactions on Knowledge and Data Engineering
Traveling to Rome: QoS Specifications for Automated Storage System Management
IWQoS '01 Proceedings of the 9th International Workshop on Quality of Service
Logging RAID - An Approach to Fast, Reliable, and Low-Cost Disk Arrays
Euro-Par '00 Proceedings from the 6th International Euro-Par Conference on Parallel Processing
My Cache or Yours? Making Storage More Exclusive
ATEC '02 Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference
Bridging the Information Gap in Storage Protocol Stacks
ATEC '02 Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference
Performance Analysis of Storage Systems
Performance Evaluation: Origins and Directions
Modeling and Dimensioning Hierarchical Storage Systems for Low-Delay Video Services
IEEE Transactions on Computers
A Novel Reordering Write Buffer to Improve Write Performance of Log-Structured File Systems
IEEE Transactions on Computers
Atomic writes for data integrity and consistency in shared storage devices for clusters
Future Generation Computer Systems - Special issue: Advanced services for clusters and internet computing
X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs
Proceedings of the 31st annual international symposium on Computer architecture
D-SPTF: decentralized request distribution in brick-based storage systems
ASPLOS XI Proceedings of the 11th international conference on Architectural support for programming languages and operating systems
ASPLOS XI Proceedings of the 11th international conference on Architectural support for programming languages and operating systems
IEEE Transactions on Parallel and Distributed Systems
ICDE '05 Proceedings of the 21st International Conference on Data Engineering
Improving storage system availability with D-GRAID
ACM Transactions on Storage (TOS)
Oasis: an architecture for simplified data management and disconnected operation
Personal and Ubiquitous Computing
WOLF--A Novel Reordering Write Buffer to Boost the Performance of Log-Structured File System
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Myriad: Cost-effective Disaster Tolerance
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Hippodrome: Running Circles Around Storage Administration
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Selecting RAID Levels for Disk Arrays
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Configuring and Scheduling an Eager-Writing Disk Array for a Transaction Processing Workload
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Systems Support for Preemptive Disk Scheduling
IEEE Transactions on Computers
Hibernator: helping disk arrays sleep through the winter
Proceedings of the twentieth ACM symposium on Operating systems principles
Proceedings of the twentieth ACM symposium on Operating systems principles
Awarded Best Student Paper! -- Improving Storage System Availability with D-GRAID
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
Atropos: A Disk Array Volume Manager for Orchestrated Use of Disks
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
Semantically-smart disk systems: past, present, and future
ACM SIGMETRICS Performance Evaluation Review - Design, implementation, and performance of storage systems
CEFT: A cost-effective, fault-tolerant parallel virtual file system
Journal of Parallel and Distributed Computing
Capsule: an energy-optimized object storage system for memory-constrained sensor devices
Proceedings of the 4th international conference on Embedded networked sensor systems
An approach to virtual allocation in storage systems
ACM Transactions on Storage (TOS)
SLAS: An efficient approach to scaling round-robin striped volumes
ACM Transactions on Storage (TOS)
Implementation of a Linux log-structured file system with a garbage collector
ACM SIGOPS Operating Systems Review
Reliability tradeoffs in personal storage systems
ACM SIGOPS Operating Systems Review
An external-memory depth-first search algorithm for general grid graphs
Theoretical Computer Science
Improving the performance of log-structured file systems with adaptive block rearrangement
Proceedings of the 2007 ACM symposium on Applied computing
Ursa minor: versatile cluster-based storage
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Database-aware semantically-smart storage
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Towards higher disk head utilization: extracting free bandwidth from busy disk drives
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
Trading capacity for performance in a disk array
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Matrix-Stripe-Cache-Based Contiguity Transform for Fragmented Writes in RAID-5
IEEE Transactions on Computers
Low-overhead byzantine fault-tolerant storage
Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
Parity lost and parity regained
FAST'08 Proceedings of the 6th USENIX Conference on File and Storage Technologies
Prefetching with adaptive cache culling for striped disk arrays
ATC'08 USENIX 2008 Annual Technical Conference on Annual Technical Conference
RAID0.5: design and implementation of a low cost disk array data protection method
The Journal of Supercomputing
O2S2: enhanced object-based virtualized storage
ACM SIGOPS Operating Systems Review
Traveling to Rome: a retrospective on the journey
ACM SIGOPS Operating Systems Review
Umbrella file system: Storage management across heterogeneous devices
ACM Transactions on Storage (TOS)
Bus Modelling in Zoned Disks RAID Storage Systems
Electronic Notes in Theoretical Computer Science (ENTCS)
DHIS: discriminating hierarchical storage
SYSTOR '09 Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference
Online reorganization of databases
ACM Computing Surveys (CSUR)
FTL design exploration in reconfigurable high-performance SSD for server applications
Proceedings of the 23rd international conference on Supercomputing
AVSS: An Adaptable Virtual Storage System
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
Ultra-low power data storage for sensor networks
ACM Transactions on Sensor Networks (TOSN)
Higher reliability redundant disk arrays: Organization, operation, and coding
ACM Transactions on Storage (TOS)
DiskReduce: RAID for data-intensive scalable computing
Proceedings of the 4th Annual Workshop on Petascale Data Storage
FRASH: Exploiting storage class memory in hybrid file system for hierarchical storage
ACM Transactions on Storage (TOS)
ATTEST: ATTributes-based Extendable STorage
Journal of Systems and Software
SSS'06 Proceedings of the 8th international conference on Stabilization, safety, and security of distributed systems
Stripe-cache: an efficient cache scheme for building multimedia oriented RAID system
HCI'07 Proceedings of the 12th international conference on Human-computer interaction: applications and services
Virtual large-scale disk system for pc-room
NBiS'07 Proceedings of the 1st international conference on Network-based information systems
Extensible block-level storage virtualization in cluster-based systems
Journal of Parallel and Distributed Computing
Optimizations of LFS with slack space recycling and lazy indirect block update
Proceedings of the 3rd Annual Haifa Experimental Systems Conference
DARC: design and evaluation of an I/O controller for data protection
Proceedings of the 3rd Annual Haifa Experimental Systems Conference
Everest: scaling down peak loads through I/O off-loading
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Removing the costs of indirection in flash-based SSDs with nameless writes
HotStorage'10 Proceedings of the 2nd USENIX conference on Hot topics in storage and file systems
Storage device performance prediction with selective bagging classification and regression tree
NPC'10 Proceedings of the 2010 IFIP international conference on Network and parallel computing
Differentiated storage services
ACM SIGOPS Operating Systems Review
FastScale: accelerate RAID scaling by minimizing data migration
FAST'11 Proceedings of the 9th USENIX conference on File and stroage technologies
WOLF: a novel reordering write buffer to boost the performance of log-structured file systems
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
Myriad: cost-effective disaster tolerance
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
Hippodrome: running circles around storage administration
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
Selecting RAID levels for disk arrays
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
Track-aligned extents: matching access patterns to disk drive characteristics
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
Configuring and scheduling an eager-writing disk array for a transaction processing workload
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
Improving storage system availability with D-GRAID
FAST'04 Proceedings of the 3rd USENIX conference on File and storage technologies
Atropos: a disk array volume manager for orchestrated use of disks
FAST'04 Proceedings of the 3rd USENIX conference on File and storage technologies
Why specialized disks for composite operations may be unnecessary
ACM SIGARCH Computer Architecture News
Hystor: making the best use of solid state drives in high performance storage systems
Proceedings of the international conference on Supercomputing
RAID level selection for heterogeneous disk arrays
Cluster Computing
YouChoose: Choosing your Storage Device as a Performance Interface to Consolidated I/O Service
ACM Transactions on Storage (TOS)
Differentiated storage services
SOSP '11 Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles
Towards reliable storage systems
Towards reliable storage systems
DPCT: distributed parity cache table for redundant parallel file system
HPCC'06 Proceedings of the Second international conference on High Performance Computing and Communications
ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part III
A RAID controller: software, hardware and embedded platform based on intel IOP321
ICESS'04 Proceedings of the First international conference on Embedded Software and Systems
Frugal storage for cloud file systems
Proceedings of the 7th ACM european conference on Computer Systems
De-indirection for flash-based SSDs with nameless writes
FAST'12 Proceedings of the 10th USENIX conference on File and Storage Technologies
SFS: random write considered harmful in solid state drives
FAST'12 Proceedings of the 10th USENIX conference on File and Storage Technologies
Hybrid aggregates: combining SSDs and HDDs in a single storage pool
ACM SIGOPS Operating Systems Review
A reliability optimization method for RAID-structured storage systems based on active data migration
Journal of Systems and Software
Design and Evaluation of a New Approach to RAID-0 Scaling
ACM Transactions on Storage (TOS)
Proceedings of the 1st Workshop on Interactions of NVM/FLASH with Operating Systems and Workloads
Janus: optimal flash provisioning for cloud storage workloads
USENIX ATC'13 Proceedings of the 2013 USENIX conference on Annual Technical Conference
HARDFS: hardening HDFS with selective and lightweight versioning
FAST'13 Proceedings of the 11th USENIX conference on File and Storage Technologies
Gecko: contention-oblivious disk arrays for cloud storage
FAST'13 Proceedings of the 11th USENIX conference on File and Storage Technologies
CRAID: online RAID upgrades using dynamic hot data reorganization
FAST'14 Proceedings of the 12th USENIX conference on File and Storage Technologies
Hi-index | 0.01 |
Configuring redundant disk arrays is a black art. To configure an array properly, a system administrator must understand the details of both the array and the workload it will support. Incorrect understanding of either, or changes in the workload over time, can lead to poor performance. We present a solution to this problem: a two-level storage hierarchy implemented inside a single disk-array controller. In the upper level of this hierarchy, two copies of active data are stored to provide full redundancy and excellent performance. In the lower level, RAID 5 parity protection is used to provide excellent storage cost for inactive data, at somewhat lower performance. The technology we describe in this article, know as HP AutoRAID, automatically and transparently manages migration of data blocks between these two levels as access patterns change. The result is a fully redundant storage system that is extremely easy to use, is suitable for a wide variety of workloads, is largely insensitive to dynamic workload changes, and performs much better than disk arrays with comparable numbers of spindles and much larger amounts of front-end RAM cache. Because the implementation of the HP AutoRAID technology is almost entirely in software, the additional hardware cost for these benefits is very small. We describe the HP AutoRAID technology in detail, provide performance data for an embodiment of it in a storage array, and summarize the results of simulation studies used to choose algorithms implemented in the array.