Scale and performance in a distributed file system
ACM Transactions on Computer Systems (TOCS)
A new approach to I/O performance evaluation: self-scaling I/O benchmarks, predicted I/O performance
SIGMETRICS '93 Proceedings of the 1993 ACM SIGMETRICS conference on Measurement and modeling of computer systems
The design and implementation of a log-structured file system
The design and implementation of a log-structured file system
File system aging—increasing the relevance of file system benchmarks
SIGMETRICS '97 Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Virtual log based file systems for a programmable disk
OSDI '99 Proceedings of the third symposium on Operating systems design and implementation
The design of a multicast-based distributed file system
OSDI '99 Proceedings of the third symposium on Operating systems design and implementation
Integrating content-based access mechanisms with hierarchical file systems
OSDI '99 Proceedings of the third symposium on Operating systems design and implementation
File system usage in Windows NT 4.0
Proceedings of the seventeenth ACM symposium on Operating systems principles
Deciding when to forget in the Elephant file system
Proceedings of the seventeenth ACM symposium on Operating systems principles
Separating key management from file system security
Proceedings of the seventeenth ACM symposium on Operating systems principles
A trace-driven analysis of the UNIX 4.2 BSD file system
Proceedings of the tenth ACM symposium on Operating systems principles
A low-bandwidth network file system
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Wide-area cooperative storage with CFS
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Why and how to benchmark XML databases
ACM SIGMOD Record
Obtaining High Performance for Storage Outsourcing
FAST '02 Proceedings of the Conference on File and Storage Technologies
PersonalRAID: Mobile Storage for Distributed and Disconnected Computers
FAST '02 Proceedings of the Conference on File and Storage Technologies
GPFS: A Shared-Disk File System for Large Computing Clusters
FAST '02 Proceedings of the Conference on File and Storage Technologies
Strong Security for Network-Attached Storage
FAST '02 Proceedings of the Conference on File and Storage Technologies
Track-Aligned Extents: Matching Access Patterns to Disk Drive Characteristics
FAST '02 Proceedings of the Conference on File and Storage Technologies
Freeblock Scheduling Outside of Disk Firmware
FAST '02 Proceedings of the Conference on File and Storage Technologies
Configuring and Scheduling an Eager-Writing Disk Array for a Transaction Processing Workload
FAST '02 Proceedings of the Conference on File and Storage Technologies
Venti: A New Approach to Archival Storage
FAST '02 Proceedings of the Conference on File and Storage Technologies
SnapMirror: File-System-Based Asynchronous Mirroring for Disaster Recovery
FAST '02 Proceedings of the Conference on File and Storage Technologies
Hippodrome: Running Circles Around Storage Administration
FAST '02 Proceedings of the Conference on File and Storage Technologies
Bridging the Information Gap in Storage Protocol Stacks
ATEC '02 Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference
Structure and Performance of the Direct Access File System
ATEC '02 Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference
Conquest: Better Performance Through a Disk/Persistent-RAM Hybrid File System
ATEC '02 Proceedings of the General Track of the annual conference on USENIX Annual Technical Conference
The Multi-Queue Replacement Algorithm for Second Level Buffer Caches
Proceedings of the General Track: 2002 USENIX Annual Technical Conference
Fast Indexing: Support for Size-Changing Algorithms in Stackable File Systems
Proceedings of the General Track: 2002 USENIX Annual Technical Conference
Design and Implementation of a Predictive File Prefetching Algorithm
Proceedings of the General Track: 2002 USENIX Annual Technical Conference
Unifying File System Protection
Proceedings of the General Track: 2002 USENIX Annual Technical Conference
Charm: An I/O-Driven Execution Strategy for High-Performance Transaction Processing
Proceedings of the General Track: 2002 USENIX Annual Technical Conference
Enhancing NFS Cross-Administrative Domain Access
Proceedings of the FREENIX Track: 2002 USENIX Annual Technical Conference
Overhauling Amd for the '00s: A Case Study of GNU Autotools
Proceedings of the FREENIX Track: 2002 USENIX Annual Technical Conference
Filesystem Performance and Scalability in Linux 2.4.17
Proceedings of the FREENIX Track: 2002 USENIX Annual Technical Conference
A Toolkit for User-Level File Systems
Proceedings of the General Track: 2002 USENIX Annual Technical Conference
Brittle Metrics in Operating Systems Research
HOTOS '99 Proceedings of the The Seventh Workshop on Hot Topics in Operating Systems
The Case for Application-Specific Benchmarking
HOTOS '99 Proceedings of the The Seventh Workshop on Hot Topics in Operating Systems
File System Benchmarks, Then, Now, and Tomorrow
MSS '01 Proceedings of the Eighteenth IEEE Symposium on Mass Storage Systems and Technologies
Long Term Distributed File Reference Tracing: Implementation and Experience
Long Term Distributed File Reference Tracing: Implementation and Experience
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Decentralized user authentication in a global file system
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Transforming policies into mechanisms with infokernel
SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Cryptographic File Systems Performance: What You Don't Know Can Hurt You
SISW '03 Proceedings of the Second IEEE International Security in Storage Workshop
The Brave Little Toaster Meets Usenet
LISA '96 Proceedings of the 10th USENIX conference on System administration
The Advancement of NFS Benchmarking: SFS 2.0
LISA '99 Proceedings of the 13th USENIX conference on System administration
New NFS Tracing Tools and Techniques for System Analysis
LISA '03 Proceedings of the 17th USENIX conference on System administration
Farsite: federated, available, and reliable storage for an incompletely trusted environment
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
Taming aggressive replication in the Pangaea wide-area file system
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
Ivy: a read/write peer-to-peer file system
OSDI '02 Proceedings of the 5th symposium on Operating systems design and implementationCopyright restrictions prevent ACM from being able to make the PDFs for this conference available for downloading
Solaris Internals (2nd Edition)
Solaris Internals (2nd Edition)
Safety, Visibility, and Performance in a Wide-Area File System
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Exploiting Inter-File Access Patterns Using Multi-Collective I/O
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Aqueduct: Online Data Migration with Performance Guarantees
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Awarded Best Student Paper! - Pond: The OceanStore Prototype
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Data Staging on Untrusted Surrogates
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Plutus: Scalable Secure File Sharing on Untrusted Storage
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Metadata Efficiency in Versioning File Systems
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
yFS: A Journaling File System Design for Handling Large Data Sets with Reduced Seeking
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Semantically-Smart Disk Systems
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Façade: Virtual Storage Devices with Performance Guarantees
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Design and Implementation of Semi-preemptible IO
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Block-Level Security for Network-Attached Disks
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Making the Most Out of Direct-Access Network Attached Storage
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Passive NFS Tracing of Email and Research Workloads
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Storage Over IP: When Does Hardware Support Help?
FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
Hibernator: helping disk arrays sleep through the winter
Proceedings of the twentieth ACM symposium on Operating systems principles
Speculative execution in a distributed file system
Proceedings of the twentieth ACM symposium on Operating systems principles
Proceedings of the twentieth ACM symposium on Operating systems principles
Proceedings of the twentieth ACM symposium on Operating systems principles
Awarded Best Paper! -- Row-Diagonal Parity for Double Disk Failure Correction
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
Awarded Best Student Paper! -- Improving Storage System Availability with D-GRAID
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
Buttress: A Toolkit for Flexible and High Fidelity I/O Benchmarking
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
A Performance Comparison of NFS and iSCSI for IP-Networked Storage
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
A Versatile and User-Oriented Versioning File System
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
Tracefs: A File System to Trace Them All
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
Atropos: A Disk Array Volume Manager for Orchestrated Use of Disks
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
Awarded Best Student Paper! -- A Framework for Building Unobtrusive Disk Maintenance Applications
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
Integrating Portable and Distributed Storage
FAST '04 Proceedings of the 3rd USENIX Conference on File and Storage Technologies
NFS tricks and benchmarking traps
ATEC '03 Proceedings of the annual conference on USENIX Annual Technical Conference
Analysis and evolution of journaling file systems
ATEC '05 Proceedings of the annual conference on USENIX Annual Technical Conference
Comparison-based file server verification
ATEC '05 Proceedings of the annual conference on USENIX Annual Technical Conference
Auto-pilot: a platform for system software benchmarking
ATEC '05 Proceedings of the annual conference on USENIX Annual Technical Conference
Energy efficient prefetching and caching
ATEC '04 Proceedings of the annual conference on USENIX Annual Technical Conference
Providing tunable consistency for a parallel file store
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Ursa minor: versatile cluster-based storage
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Journal-guided resynchronization for software RAID
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Secure deletion for a versioning file system
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
On multidimensional data and modern disks
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Database-aware semantically-smart storage
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
A scalable and high performance software iSCSI implementation
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
TBBT: scalable and accurate trace replay for file server evaluation
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Accurate and efficient replaying of file system traces
FAST'05 Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies - Volume 4
Latency management in storage systems
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
Self-securing storage: protecting data in compromised system
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
Fast and secure distributed read-only file system
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
Trading capacity for performance in a disk array
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
Interposed request routing for scalable network storage
OSDI'00 Proceedings of the 4th conference on Symposium on Operating System Design & Implementation - Volume 4
Boxwood: abstractions as the foundation for storage infrastructure
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Secure untrusted data repository (SUNDR)
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Energy-efficiency and storage flexibility in the blue file system
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Program-counter-based pattern classification in buffer caching
OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
Provenance-aware storage systems
ATEC '06 Proceedings of the annual conference on USENIX '06 Annual Technical Conference
Thresher: an efficient storage manager for copy-on-write snapshots
ATEC '06 Proceedings of the annual conference on USENIX '06 Annual Technical Conference
A comparison of file system workloads
ATEC '00 Proceedings of the annual conference on USENIX Annual Technical Conference
FiST: a language for stackable file systems
ATEC '00 Proceedings of the annual conference on USENIX Annual Technical Conference
Journaling versus soft updates: asynchronous meta-data protection in file systems
ATEC '00 Proceedings of the annual conference on USENIX Annual Technical Conference
A five-year study of file-system metadata
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Proportional-share scheduling for distributed storage systems
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Argon: performance insulation for shared storage servers
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Strong accountability for network storage
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Design and implementation of verifiable audit trails for a versioning file system
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Data ONTAP GX: a scalable storage cluster
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Trace: parallel trace replay with approximate causal events
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Nache: design and implementation of a caching proxy for NFSv4
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
TFS: a transparent file system for contributory storage
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
Cobalt: separating content distribution from authorization in distributed file systems
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
PARAID: a gear-shifting power-aware RAID
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
FAST '07 Proceedings of the 5th USENIX conference on File and Storage Technologies
PenguinoMeter: a new file-I/O benchmark for Linux®
ALS '01 Proceedings of the 5th annual Linux Showcase & Conference - Volume 5
Observing the effects of multi-zone disks
ATEC '97 Proceedings of the annual conference on USENIX Annual Technical Conference
Operation-based update propagation in a mobile file system
ATEC '99 Proceedings of the annual conference on USENIX Annual Technical Conference
Extending file systems using stackable templates
ATEC '99 Proceedings of the annual conference on USENIX Annual Technical Conference
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
Operating system profiling via latency analysis
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
EnsemBlue: integrating distributed storage and consumer electronics
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
Ceph: a scalable, high-performance distributed file system
OSDI '06 Proceedings of the 7th symposium on Operating systems design and implementation
LADDIS: the next generation in NFS file server benchmarking
Usenix-stc'93 Proceedings of the USENIX Summer 1993 Technical Conference on Summer technical conference - Volume 1
Umbrella file system: Storage management across heterogeneous devices
ACM Transactions on Storage (TOS)
Higher reliability redundant disk arrays: Organization, operation, and coding
ACM Transactions on Storage (TOS)
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
Lock management in blue whale file system
Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human
Oracle SecureFiles: prepared for the digital deluge
Proceedings of the VLDB Endowment
NCQ vs. I/O scheduler: Preventing unexpected misbehaviors
ACM Transactions on Storage (TOS)
Towards a secure and efficient system for end-to-end provenance
TAPP'10 Proceedings of the 2nd conference on Theory and practice of provenance
Paxos replicated state machines as the basis of a high-performance data store
Proceedings of the 8th USENIX conference on Networked systems design and implementation
Sustainable predictive storage management: on-line grouping for energy and latency reduction
Proceedings of the 4th Annual International Conference on Systems and Storage
Benchmarking file system benchmarking: it *IS* rocket science
HotOS'13 Proceedings of the 13th USENIX conference on Hot topics in operating systems
Understanding and Improving Computational Science Storage Access through Continuous Characterization
ACM Transactions on Storage (TOS)
An efficient multi-tier tablet server storage architecture
Proceedings of the 2nd ACM Symposium on Cloud Computing
A caching-oriented management design for the performance enhancement of solid-state drives
ACM Transactions on Storage (TOS)
Robust benchmarking for archival storage tiers
Proceedings of the sixth workshop on Parallel Data Storage
Analysis of Workload Behavior in Scientific and Historical Long-Term Data Repositories
ACM Transactions on Storage (TOS)
Extracting flexible, replayable models from large block traces
FAST'12 Proceedings of the 10th USENIX conference on File and Storage Technologies
Optimizing Storage Performance for VM-Based Mobile Computing
ACM Transactions on Computer Systems (TOCS)
SAW: system-assisted wear leveling on the write endurance of NAND flash devices
Proceedings of the 50th Annual Design Automation Conference
Hi-index | 0.00 |
Benchmarking is critical when evaluating performance, but is especially difficult for file and storage systems. Complex interactions between I/O devices, caches, kernel daemons, and other OS components result in behavior that is rather difficult to analyze. Moreover, systems have different features and optimizations, so no single benchmark is always suitable. The large variety of workloads that these systems experience in the real world also adds to this difficulty. In this article we survey 415 file system and storage benchmarks from 106 recent papers. We found that most popular benchmarks are flawed and many research papers do not provide a clear indication of true performance. We provide guidelines that we hope will improve future performance evaluations. To show how some widely used benchmarks can conceal or overemphasize overheads, we conducted a set of experiments. As a specific example, slowing down read operations on ext2 by a factor of 32 resulted in only a 2--5% wall-clock slowdown in a popular compile benchmark. Finally, we discuss future work to improve file system and storage benchmarking.