Measurements of a distributed file system
SOSP '91 Proceedings of the thirteenth ACM symposium on Operating systems principles
The design and implementation of a log-structured file system
ACM Transactions on Computer Systems (TOCS)
RAID-II: a high-bandwidth network file server
ISCA '94 Proceedings of the 21st annual international symposium on Computer architecture
The Zebra striped network file system
ACM Transactions on Computer Systems (TOCS)
Input/output characteristics of scalable parallel applications
Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
The Vesta parallel file system
ACM Transactions on Computer Systems (TOCS)
File-Access Characteristics of Parallel Scientific Workloads
IEEE Transactions on Parallel and Distributed Systems
Dynamic file-access characteristics of a production parallel scientific workload
Proceedings of the 1994 ACM/IEEE conference on Supercomputing
Characterizing parallel file-access patterns on a large-scale multiprocessor
IPPS '95 Proceedings of the 9th International Symposium on Parallel Processing
Workload Characterization of Input/Output Intensive Parallel Applications
Proceedings of the 9th International Conference on Computer Performance Evaluation: Modelling Techniques and Tools
Data Sieving and Collective I/O in ROMIO
FRONTIERS '99 Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation
I/O Requirements of Scientific Applications: An Evolutionary View
HPDC '96 Proceedings of the 5th IEEE International Symposium on High Performance Distributed Computing
Exporting Storage Systems in a Scalable Manner with pNFS
MSST '05 Proceedings of the 22nd IEEE / 13th NASA Goddard Conference on Mass Storage Systems and Technologies
GPFS: A Shared-Disk File System for Large Computing Clusters
FAST '02 Proceedings of the 1st USENIX Conference on File and Storage Technologies
Direct-pNFS: scalable, transparent, and versatile access to parallel file systems
Proceedings of the 16th international symposium on High performance distributed computing
pNFS/PVFS2 over InfiniBand: early experiences
PDSW '07 Proceedings of the 2nd international workshop on Petascale data storage: held in conjunction with Supercomputing '07
DARAW: a new write buffer to improve parallel I/O energy-efficiency
Proceedings of the 2009 ACM symposium on Applied Computing
Lock management in blue whale file system
Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human
pNFS, POSIX, and MPI-IO: a tale of three semantics
Proceedings of the 4th Annual Workshop on Petascale Data Storage
Design and implementation of parallel file aggregation mechanism
Proceedings of the 1st International Workshop on Runtime and Operating Systems for Supercomputers
Performance optimization of small file i/o with adaptive migration strategy in cluster file system
HPCA'09 Proceedings of the Second international conference on High Performance Computing and Applications
Improving Bandwidth Efficiency for Consistent Multistream Storage
ACM Transactions on Storage (TOS)
| Hi-index | 0.00 |
Workload characterization studies highlight the prevalence of small and sequential data requests in scientific applications. Parallel file systems excel at large data transfers but sometimes at the expense of small I/O performance. pNFS is an NFSv4.1 high-performance enhancement that provides direct storage access to parallel file systems while preserving NFSv4 operating system and hardware platform independence. This paper demonstrates that distributed file systems can increase write throughput to parallel data stores---regardless of file size---by overcoming parallel file system inefficiencies. We also show how pNFS can improve the overall write performance of parallel file systems by using direct, parallel I/O for large write requests and a distributed file system for small write requests. We describe our pNFS prototype and present experiments demonstrating the performance improvements.