Server-directed collective I/O in Panda
Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
A trace-driven comparison of algorithms for parallel prefetching and caching
OSDI '96 Proceedings of the second USENIX symposium on Operating systems design and implementation
Implementation of collective I/O in the Intel Paragon parallel file system: initial experiences
ICS '97 Proceedings of the 11th international conference on Supercomputing
Automatic I/O hint generation through speculative execution
OSDI '99 Proceedings of the third symposium on Operating systems design and implementation
On implementing MPI-IO portably and with high performance
Proceedings of the sixth workshop on I/O in parallel and distributed systems
Optimizing noncontiguous accesses in MPI – IO
Parallel Computing
MPI-IO/GPFS, an optimized implementation of MPI-IO on top of GPFS
Proceedings of the 2001 ACM/IEEE conference on Supercomputing
PC-OPT: Optimal Offline Prefetching and Caching for Parallel I/O Systems
IEEE Transactions on Computers
Improving Collective I/O Performance Using Threads
IPPS '99/SPDP '99 Proceedings of the 13th International Symposium on Parallel Processing and the 10th Symposium on Parallel and Distributed Processing
Towards a High-Performance Implementation of MPI-IO on Top of GPFS
Euro-Par '00 Proceedings from the 6th International Euro-Par Conference on Parallel Processing
Workload Characterization of Input/Output Intensive Parallel Applications
Proceedings of the 9th International Conference on Computer Performance Evaluation: Modelling Techniques and Tools
An Abstract-Device Interface for Implementing Portable Parallel-I/O Interfaces
FRONTIERS '96 Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation
Data Sieving and Collective I/O in ROMIO
FRONTIERS '99 Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation
Improving MPI-IO Output Performance with Active Buffering Plus Threads
IPDPS '03 Proceedings of the 17th International Symposium on Parallel and Distributed Processing
MPI-IO on a Parallel File System for Cluster of Workstations
IWCC '99 Proceedings of the 1st IEEE Computer Society International Workshop on Cluster Computing
Parallel netCDF: A High-Performance Scientific I/O Interface
Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Integrating Logical and Physical File Models in the MPI-IO Implementation for "Clusterfile"
CCGRID '06 Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid
Collective caching: application-aware client-side file caching
HPDC '05 Proceedings of the High Performance Distributed Computing, 2005. HPDC-14. Proceedings. 14th IEEE International Symposium
View-Based Collective I/O for MPI-IO
CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Hiding I/O latency with pre-execution prefetching for parallel applications
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Parallel I/O prefetching using MPI file caching and I/O signatures
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Latency Hiding File I/O for Blue Gene Systems
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
ViMPIOS, a "truly" portable MPI-IO implementation
EURO-PDP'00 Proceedings of the 8th Euromicro conference on Parallel and distributed processing
GPFS: a shared-disk file system for large computing clusters
FAST'02 Proceedings of the 1st USENIX conference on File and storage technologies
Cooperative write-behind data buffering for MPI i/o
PVM/MPI'05 Proceedings of the 12th European PVM/MPI users' group conference on Recent Advances in Parallel Virtual Machine and Message Passing Interface
Improving collective I/O performance using pipelined two-phase I/O
Proceedings of the 2012 Symposium on High Performance Computing
| Hi-index | 0.00 |
In this paper we present the implementation of an open-source MPI-IO interface for the General Parallel File System (GPFS). Our solution includes the design and implementation of GPFS-based write-back and prefetching modules, which have been integrated in ROMIO. A collective file write strategy based on GPFS data-shipping, and a view-based collective I/O mechanism, relying on GPFS mechanisms, are at the core of the novel optimizations proposed in this paper. View-based collective I/O includes a thread-based flushing method implementing a write-back policy for latency hiding, and a prefetching method, based on GPFS hints, to increase small read access performance. Performance evaluations show that our implementation achieves high-performance and hides the latency of file accesses through the combination of view-based collective file accesses, and the overlapping of computation, communication and I/O. This is especially true for collective and small-size access patterns, which are very frequent in parallel scientific applications.