Improving the average response time in collective I/O

Authors:
Chen Jin;Saba Sehrish;Wei-keng Liao;Alok Choudhary;Karen Schuchardt
Affiliations:
Northwestern University, Evanston, IL;Northwestern University, Evanston, IL;Northwestern University, Evanston, IL;Northwestern University, Evanston, IL;Pacific Northwest National Laboratory, Richland, WA
Venue:
EuroMPI'11 Proceedings of the 18th European MPI Users' Group conference on Recent advances in the message passing interface
Year:
2011

Citing 10
Cited 0

Applying randomized edge coloring algorithms to distributed communication: an experimental study

Proceedings of the seventh annual ACM symposium on Parallel algorithms and architectures
Server-directed collective I/O in Panda

Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
Disk-directed I/O for MIMD multiprocessors

ACM Transactions on Computer Systems (TOCS)
Data Sieving and Collective I/O in ROMIO

FRONTIERS '99 Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation
Parallel netCDF: A High-Performance Scientific I/O Interface

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Efficient Distributed Algorithms for Parallel I/O Scheduling

ICPADS '05 Proceedings of the 11th International Conference on Parallel and Distributed Systems - Volume 01
Dynamically adapting file domain partitioning methods for collective I/O based on underlying parallel file system locking protocols

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Performance Evaluation of Collective Write Algorithms in MPI I/O

ICCS '09 Proceedings of the 9th International Conference on Computational Science: Part I
On evaluating decentralized parallel I/O scheduling strategies for parallel file systems

VECPAR'06 Proceedings of the 7th international conference on High performance computing for computational science
Design and Evaluation of MPI File Domain Partitioning Methods under Extent-Based File Locking Protocol

IEEE Transactions on Parallel and Distributed Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

In collective I/O, MPI processes exchange requests so that the rearranged requests can result in the shortest file system access time. Scheduling the exchange sequence determines the response time of participating processes. Existing implementations that simply follow the increasing order of file offsets do not necessary produce the best performance. To minimize the average response time, we propose three scheduling algorithms that consider the number of processes per file stripe and the number of accesses per process. Our experimental results demonstrate improvements of up to 50% in the average response time using two synthetic benchmarks and a high-resolution climate application.