Data Sieving and Collective I/O in ROMIO
FRONTIERS '99 Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation
LIVE data workspace: A flexible, dynamic and extensible platform for petascale applications
CLUSTER '07 Proceedings of the 2007 IEEE International Conference on Cluster Computing
Plasma fusion code coupling using scalable I/O services and scientific workflows
Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science
Experiments with Memory-to-Memory Coupling for End-to-End Fusion Simulation Workflows
CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
DataSpaces: an interaction and coordination framework for coupled simulation workflows
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
An HDF5 MPI virtual file driver for parallel in-situ post-processing
EuroMPI'10 Proceedings of the 17th European MPI users' group meeting conference on Recent advances in the message passing interface
Enabling event tracing at leadership-class scale through I/O forwarding middleware
Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing
Combining in-situ and in-transit processing to enable extreme-scale scientific analysis
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Optimizing I/O forwarding techniques for extreme-scale event tracing
Cluster Computing
| Hi-index | 0.00 |
Large scale simulations of complex physics phenomena have long run times and generate massive amounts of data. Saving this data to external storage systems or transferring it to remote locations for analysis is a costly operation that quickly becomes a performance bottleneck. In this paper, we present DART (Decoupled and Asynchronous Remote Transfers), an efficient data transfer substrate that effectively minimizes the data I/O overhead on the running simulations. DART is a thin software layer built on RDMA technology to enable fast, low-overhead and asynchronous access to data from a running simulation, and support high-throughput, low-latency data transfers.