Using MPI file caching to improve parallel write performance for large-scale scientific applications
Proceedings of the 2007 ACM/IEEE conference on Supercomputing
Scaling parallel I/O performance through I/O delegate and caching system
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Hiding I/O latency with pre-execution prefetching for parallel applications
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Parallel I/O prefetching using MPI file caching and I/O signatures
Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Future Generation Computer Systems
Implementation and Evaluation of an MPI-IO Interface for GPFS in ROMIO
Proceedings of the 15th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface
A collective I/O implementation based on inspector---executor paradigm
The Journal of Supercomputing
Data Locality Aware Strategy for Two-Phase Collective I/O
High Performance Computing for Computational Science - VECPAR 2008
Latency Hiding File I/O for Blue Gene Systems
CCGRID '09 Proceedings of the 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid
Implementation and Evaluation of File Write-Back and Prefetching for MPI-IO Over GPFS
International Journal of High Performance Computing Applications
A Scalable Message Passing Interface Implementation of an Ad-Hoc Parallel I/o system
International Journal of High Performance Computing Applications
A cost-intelligent application-specific data layout scheme for parallel file systems
Proceedings of the 20th international symposium on High performance distributed computing
Software-directed data access scheduling for reducing disk energy consumption
Proceedings of the 20th international symposium on High performance distributed computing
| Hi-index | 0.00 |
Parallel file subsystems in today's high-performance computers adopt many I/O optimization strategies that were designed for distributed systems. These strategies, for instance client-side file caching, treat each I/O request process independently, due to the consideration that clients are unlikely related with each other in a distributed environment. However, it is inadequate to apply such strategies directly in the high-performance computers where most of the I/O requests come from the processes that work on the same parallel applications. We believe that client-side caching could perform more effectively if the caching subsystem is aware of the process scope of an application and regards all the application processes as a single client. In this paper, we propose the idea of "collective caching" which coordinates the application processes to manage cache data and achieve cache coherence without involving the I/O servers. To demonstrate this idea, we implemented a collective caching subsystem at user space as a library, which can be incorporated into any message passing interface implementation to increase its portability. The performance evaluation is presented with three I/O benchmarks on an IBM SP using its native parallel file system, GPFS. Our results show significant performance enhancement obtained by collective caching over the traditional approaches.