Characterization and modeling of PIDX parallel I/O for performance optimization
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
| Hi-index | 0.00 |
I/O is often a limiting factor for HPC applications. Although well-tuned codes have shown good I/O throughput compared to the theoretical maximum, the majority of applications use default parallel I/O parameter values and achieve poor performance. We have built an extensible framework for benchmark-guided auto-tuning of HDF5, MPI-IO, and Lustre parameters. The framework includes three main components. H5AutoTuner uses a control file to adjust I/O parameters without changing or recompiling the application. H5PerfCapture records performance metrics for HDF5 and MPI-IO. H5Evolve uses genetic algorithms to explore the parameter search space until well-performing values are identified. Early results for three HDF5 application-based I/O benchmarks on two different HPC systems have shown 3.3x -- 16.7x speedup using auto-tuned parameters compared to default values. Our auto-tuning framework can improve I/O performance without hands-on optimization and also provides a general platform for exploring parallel I/O behavior.