Efficient input and output for scientific simulations
Proceedings of the sixth workshop on I/O in parallel and distributed systems
Integrating collective I/O and cooperative caching into the "clusterfile" parallel file system
Proceedings of the 18th annual international conference on Supercomputing
Adaptive parallel I/O scheduling algorithm for multiprogrammed systems
Future Generation Computer Systems - Parallel input/output management techniques (PIOMT) in cluster and grid computing
Adaptive parallel I/O scheduling algorithm for multiprogrammed systems
Future Generation Computer Systems - Parallel input/output management techniques (PIOMT) in cluster and grid computing
| Hi-index | 0.00 |
Fine grained data distributions are widely used to balance computational loads across compute processes in parallel scientific applications. When a fine grained data distribution is used in memory, performance of I/O intensive applications can be limited not only by disk speed but also by message passing, because a large number of small messages may be generated by the implementation strategy used in the underlying parallel file system or parallel I/O library. Combining (or packetizing) a set of small messages into a large message is generally known to speed up parallel I/O. However, overall I/O performance is affected not only by small messages but also by other factors like cyclic block size and interconnect characteristics. We describe small message combination and communication scheduling for fine grained data distributions in the Panda parallel I/O library and analyze I/O performance on parallel platforms having different interconnects: IBM SP2, IBM workstation cluster connected by FDDI and Pentium II cluster connected by Myrinet.