Caching the writeback policies in parallel file systems
Journal of Parallel and Distributed Computing - Special issue on parallel I/O systems
Improved parallel I/O via a two-phase run-time access strategy
ACM SIGARCH Computer Architecture News - Special issue on input/output in parallel computer systems
Server-directed collective I/O in Panda
Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
Tuning the performance of I/O-intensive parallel applications
Proceedings of the fourth workshop on I/O in parallel and distributed systems: part of the federated computing research conference
ENWRICH: a compute-processor write caching scheme for parallel file systems
Proceedings of the fourth workshop on I/O in parallel and distributed systems: part of the federated computing research conference
An interprocedural framework for placement of asynchronous I/O operations
ICS '96 Proceedings of the 10th international conference on Supercomputing
The Galley parallel file system
Parallel Computing - Special double issue: parallel I/O
Enhancing disk-directed I/O for fine-grained redistribution of file data
Parallel Computing - Special double issue: parallel I/O
Remote I/O: fast access to distant storage
Proceedings of the fifth workshop on I/O in parallel and distributed systems
GASS: a data movement and access service for wide area computing systems
Proceedings of the sixth workshop on I/O in parallel and distributed systems
Enhancing Data Migration Performance via Parallel Data Compression
IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Faster Collective Output through Active Buffering
IPDPS '02 Proceedings of the 16th International Parallel and Distributed Processing Symposium
Improving Collective I/O Performance Using Threads
IPPS '99/SPDP '99 Proceedings of the 13th International Symposium on Parallel Processing and the 10th Symposium on Parallel and Distributed Processing
Data Sieving and Collective I/O in ROMIO
FRONTIERS '99 Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation
Autopilot: Adaptive Control of Distributed Applications
HPDC '98 Proceedings of the 7th IEEE International Symposium on High Performance Distributed Computing
Disk-directed I/O for MIMD Multiprocessors
Disk-directed I/O for MIMD Multiprocessors
High-Level Buffering for Hiding Periodic Output Cost in Scientific Simulations
IEEE Transactions on Parallel and Distributed Systems
Constructing collaborative desktop storage caches for large scientific datasets
ACM Transactions on Storage (TOS)
Coupling prefix caching and collective downloads for remote dataset access
Proceedings of the 20th annual international conference on Supercomputing
Recovering transient data: automated on-demand data reconstruction and offloading for supercomputers
ACM SIGOPS Operating Systems Review
| Hi-index | 0.00 |
Scientific simulations running on parallel platforms output intermediate data periodically, typically moving the output to a remote machine for visualization. Due to the large data size and slow network, improvements in output and migration performance can significantly reduce simulation turnaround time. In this paper, we propose a novel simulation execution environment that integrates active buffering and compressed migration to hide and reduce the cost of output and migration. Our implementation removes previous shortcomings of active buffering and of compressed migration, and our experiments with real world data sets show that integrating the two techniques brings comparable application-visible I/O performance and much more flexibility compared to compressed migration without active buffering. The performance advantage will increase with slower-to-write file formats such as HDF.