Design and implementation of the Wisconsin storage system
Software—Practice & Experience
4.2BSD and 4.3BSD as examples of the UNIX system
ACM Computing Surveys (CSUR) - The MIT Press scientific computation series
Scale and performance in a distributed file system
ACM Transactions on Computer Systems (TOCS)
Communications of the ACM
A case for redundant arrays of inexpensive disks (RAID)
SIGMOD '88 Proceedings of the 1988 ACM SIGMOD international conference on Management of data
Parity striping of disc arrays: low-cost reliable storage with acceptable throughput
Proceedings of the sixteenth international conference on Very large databases
Measurements of a distributed file system
SOSP '91 Proceedings of the thirteenth ACM symposium on Operating systems principles
Input/output behavior of supercomputing applications
Proceedings of the 1991 ACM/IEEE conference on Supercomputing
Concurrent file operations in a high performance
Proceedings of the 1992 ACM/IEEE conference on Supercomputing
The Zebra striped network file system
SOSP '93 Proceedings of the fourteenth ACM symposium on Operating systems principles
The high performance storage system
Proceedings of the 1993 ACM/IEEE conference on Supercomputing
Striping in large tape libraries
Proceedings of the 1993 ACM/IEEE conference on Supercomputing
A static analysis of I/O characteristics of scientific applications in a production workload
Proceedings of the 1993 ACM/IEEE conference on Supercomputing
Design and Evaluation of primitives for Parallel I/O
Proceedings of the 1993 ACM/IEEE conference on Supercomputing
Applications-driven parallel I/O
Proceedings of the 1993 ACM/IEEE conference on Supercomputing
Parallel access to files in the Vesta file system
Proceedings of the 1993 ACM/IEEE conference on Supercomputing
Practical prefetching techniques for multiprocessor file systems
Distributed and Parallel Databases - Selected papers from the first international conference on parallel and distributed information systems
Compiler and runtime support for out-of-core HPF programs
ICS '94 Proceedings of the 8th international conference on Supercomputing
RAID-II: a high-bandwidth network file server
ISCA '94 Proceedings of the 21st annual international symposium on Computer architecture
A trace-driven analysis of the UNIX 4.2 BSD file system
Proceedings of the tenth ACM symposium on Operating systems principles
Maximizing performance in a striped disk array
ISCA '90 Proceedings of the 17th annual international symposium on Computer Architecture
The COMFORT project: project synopsis (work in progress)
PDIS '93 Proceedings of the second international conference on Parallel and distributed information systems
Microprocessor file system interfaces
PDIS '93 Proceedings of the second international conference on Parallel and distributed information systems
Operating System Concepts, 4th Ed.
Operating System Concepts, 4th Ed.
Transaction Processing: Concepts and Techniques
Transaction Processing: Concepts and Techniques
Dynamic file-access characteristics of a production parallel scientific workload
Proceedings of the 1994 ACM/IEEE conference on Supercomputing
Dynamic I/O characterization of I/O intensive scientific applications
Proceedings of the 1994 ACM/IEEE conference on Supercomputing
The Gamma Database Machine Project
IEEE Transactions on Knowledge and Data Engineering
Disk Mirroring with Alternating Deferred Updates
VLDB '93 Proceedings of the 19th International Conference on Very Large Data Bases
Dynamic Memory Allocation for Multiple-Query Workloads
VLDB '93 Proceedings of the 19th International Conference on Very Large Data Bases
The LOCUS distributed operating system
SOSP '83 Proceedings of the ninth ACM symposium on Operating systems principles
Throughput of Existing Multiprocessor File Systems (An Informal Study)
Throughput of Existing Multiprocessor File Systems (An Informal Study)
RAMA: easy access to a high-bandwidth massively parallel file system
TCON'95 Proceedings of the USENIX 1995 Technical Conference Proceedings
The SPIFFI scalable video-on-demand system
SIGMOD '95 Proceedings of the 1995 ACM SIGMOD international conference on Management of data
Distributed schedule management in the Tiger video fileserver
Proceedings of the sixteenth ACM symposium on Operating systems principles
ACM Transactions on Computer Systems (TOCS)
Disk Resident Arrays: An Array-Oriented I/O Library for Out-Of-Core Computations
FRONTIERS '96 Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation
Implementing MPI-IO Atomic Mode and Shared File Pointers Using MPI One-Sided Communication
International Journal of High Performance Computing Applications
Lachesis: robust database storage management based on device-specific performance characteristics
VLDB '03 Proceedings of the 29th international conference on Very large data bases - Volume 29
Mapping functions and data redistribution for parallel files
The Journal of Supercomputing
On evaluating decentralized parallel I/O scheduling strategies for parallel file systems
VECPAR'06 Proceedings of the 7th international conference on High performance computing for computational science
Portable and scalable MPI shared file pointers
EuroMPI'11 Proceedings of the 18th European MPI Users' Group conference on Recent advances in the message passing interface
Implementing MPI-IO shared file pointers without file system support
PVM/MPI'05 Proceedings of the 12th European PVM/MPI users' group conference on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Optimizing distributed data access in grid environments by using artificial intelligence techniques
ISPA'07 Proceedings of the 5th international conference on Parallel and Distributed Processing and Applications
| Hi-index | 0.00 |
This paper presents the design and performance of SPIFFI, a scalable high-performance parallel file system intended for use by extremely I/O intensive applications including "Grand Challenge" scientific applications and multimedia systems. This paper contains experimental results from a SPIFFI prototype on a 64 node/64 disk Intel Paragon. The results show that SPIFFI provides high performance and linear scaleup on real hardware. The paper also explains how shared file pointers (i.e., file pointers that are shared by multiple processes) can simplify the design of a parallel application. By sequentializing I/O accesses and by providing dynamic I/O load balancing, a shared file pointer may even improve an application's performance.This paper also presents the predictions of a SPIFFI simulator that we validated using the prototype. The simulator results show that SPIFFI continues to provide high performance even when it is scaled to configurations with as many as 128 disks or 256 compute nodes.