End-to-end data integrity for file systems: a ZFS case study

Authors:
Yupu Zhang;Abhishek Rajimwale;Andrea C. Arpaci-Dusseau;Remzi H. Arpaci-Dusseau
Affiliations:
Computer Sciences Department, University of Wisconsin-Madison;Computer Sciences Department, University of Wisconsin-Madison;Computer Sciences Department, University of Wisconsin-Madison;Computer Sciences Department, University of Wisconsin-Madison
Venue:
FAST'10 Proceedings of the 8th USENIX conference on File and storage technologies
Year:
2010

Citing 34
Cited 13

A case for redundant arrays of inexpensive disks (RAID)

SIGMOD '88 Proceedings of the 1988 ACM SIGMOD international conference on Management of data
Fault Injection Experiments Using FIAT

IEEE Transactions on Computers
FINE: A Fault Injection and Monitoring Environment for Tracing the UNIX System Behavior Under Faults

IEEE Transactions on Software Engineering - Special issue on software reliability
Hive: fault containment for shared-memory multiprocessors

SOSP '95 Proceedings of the fifteenth ACM symposium on Operating systems principles
Field testing for cosmic ray soft errors in semiconductor memories

IBM Journal of Research and Development - Special issue: terrestrial cosmic rays and soft errors
Bugs as deviant behavior: a general approach to inferring errors in systems code

SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
An empirical study of operating systems errors

SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
A system and language for building system-specific, static analyses

PLDI '02 Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation
Increasing relevance of memory hardware errors: a case for recoverable programming models

EW 9 Proceedings of the 9th workshop on ACM SIGOPS European workshop: beyond the PC: new challenges for the operating system
Xception: A Technique for the Experimental Evaluation of Dependability in Modern Computers

IEEE Transactions on Software Engineering
A Digital Signature Based on a Conventional Encryption Function

CRYPTO '87 A Conference on the Theory and Applications of Cryptographic Techniques on Advances in Cryptology
Unifying File System Protection

Proceedings of the General Track: 2002 USENIX Annual Technical Conference
Measuring Fault Tolerance with the FTAPE Fault Injection Tool

MMB '95 Proceedings of the 8th International Conference on Modelling Techniques and Tools for Computer Performance Evaluation: Quantitative Evaluation of Computing and Communication Systems
CSSV: towards a realistic tool for statically detecting all buffer overflows in C

PLDI '03 Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation
Error-correcting codes for semiconductor memories

ISCA '84 Proceedings of the 11th annual international symposium on Computer architecture
ARCHER: using symbolic, path-sensitive analysis to detect memory access errors

Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering
Improving the reliability of commodity operating systems

SOSP '03 Proceedings of the nineteenth ACM symposium on Operating systems principles
Commercial Fault Tolerance: A Tale of Two Systems

IEEE Transactions on Dependable and Secure Computing
Disk Scrubbing in Large Archival Storage Systems

MASCOTS '04 Proceedings of the The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems
SafeMem: Exploiting ECC-Memory for Detecting Memory Leaks and Memory Corruption During Production Runs

HPCA '05 Proceedings of the 11th International Symposium on High-Performance Computer Architecture
Soft Errors in Advanced Computer Systems

IEEE Design & Test
ARC: A Self-Tuning, Low Overhead Replacement Cache

FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
More Than an Interface---SCSI vs. ATA

FAST '03 Proceedings of the 2nd USENIX Conference on File and Storage Technologies
IRON file systems

Proceedings of the twentieth ACM symposium on Operating systems principles
DieHard: probabilistic memory safety for unsafe languages

Proceedings of the 2006 ACM SIGPLAN conference on Programming language design and implementation
Dependability Analysis of Virtual Memory Systems

DSN '06 Proceedings of the International Conference on Dependable Systems and Networks
An analysis of latent sector errors in disk drives

Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
EXPLODE: a lightweight, general system for finding serious storage system errors

OSDI '06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7
The effects of metadata corruption on nfs

Proceedings of the 2007 ACM workshop on Storage security and survivability
A memory soft error measurement on production systems

ATC'07 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference
Parity lost and parity regained

FAST'08 Proceedings of the 6th USENIX Conference on File and Storage Technologies
An analysis of data corruption in the storage stack

FAST'08 Proceedings of the 6th USENIX Conference on File and Storage Technologies
DRAM errors in the wild: a large-scale field study

Proceedings of the eleventh international joint conference on Measurement and modeling of computer systems
SQCK: a declarative file system checker

OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation

Understanding latent sector errors and how to protect against them

ACM Transactions on Storage (TOS)
Keeping bits safe: how hard can it be?

Communications of the ACM
Keeping Bits Safe: How Hard Can It Be?

Queue - Storage
Warding off the dangers of data corruption with amulet

Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
Using declarative invariants for protecting file-system integrity

PLOS '11 Proceedings of the 6th Workshop on Programming Languages and Operating Systems
Recon: verifying file system consistency at runtime

FAST'12 Proceedings of the 10th USENIX conference on File and Storage Technologies
Consistency without ordering

FAST'12 Proceedings of the 10th USENIX conference on File and Storage Technologies
Recon: Verifying file system consistency at runtime

ACM Transactions on Storage (TOS)
*-Box: towards reliability and consistency in dropbox-like file synchronization services

HotStorage'13 Proceedings of the 5th USENIX conference on Hot Topics in Storage and File Systems
Efficient transactions for parallel data movement

PDSW '13 Proceedings of the 8th Parallel Data Storage Workshop
Ffsck: The Fast File-System Checker

ACM Transactions on Storage (TOS)
Ffsck: the fast file system checker

FAST'13 Proceedings of the 11th USENIX conference on File and Storage Technologies
Checking the integrity of transactional mechanisms

FAST'14 Proceedings of the 12th USENIX conference on File and Storage Technologies

Quantified Score

Hi-index	0.02

Visualization

Abstract

We present a study of the effects of disk and memory corruption on file system data integrity. Our analysis focuses on Sun's ZFS, a modern commercial offering with numerous reliability mechanisms. Through careful and thorough fault injection, we show that ZFS is robust to a wide range of disk faults. We further demonstrate that ZFS is less resilient to memory corruption, which can lead to corrupt data being returned to applications or system crashes. Our analysis reveals the importance of considering both memory and disk in the construction of truly robust file and storage systems.