An application-level parallel I/O library for Earth system models

Authors:
John M. Dennis;Jim Edwards;Ray Loy;Robert Jacob;Arthur A. Mirin;Anthony P. Craig;Mariana Vertenstein
Affiliations:
National Center for Atmospheric Research, Computer and Information Systems Laboratory, Boulder, CO, USA;National Center for Atmospheric Research Earth System Laboratory, USA;Argonne National Laboratory, Mathematics and Computer Science Division, USA;Argonne National Laboratory, Mathematics and Computer Science Division, USA;Lawrence Livermore National Laboratory, Center for Applied Scientific Computing, USA;National Center for Atmospheric Research Earth System Laboratory, USA;National Center for Atmospheric Research Earth System Laboratory, USA
Venue:
International Journal of High Performance Computing Applications
Year:
2012

Citing 9
Cited 6

Parallel netCDF: A High-Performance Scientific I/O Interface

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Practical performance portability in the Parallel Ocean Program (POP): Research Articles

Concurrency and Computation: Practice & Experience - The High Performance Architectural Challenge: Mass Market versus Proprietary Components?
Vectorizing the Community Land Model

International Journal of High Performance Computing Applications
The Model Coupling Toolkit: A New Fortran90 Toolkit for Building Multiphysics Parallel Coupled Models

International Journal of High Performance Computing Applications
M X N Communication and Parallel Interpolation in Community Climate System Model Version 3 Using the Model Coupling Toolkit

International Journal of High Performance Computing Applications
Efficient Parallel I/O in Community Atmosphere Model (CAM)

International Journal of High Performance Computing Applications
Dynamically adapting file domain partitioning methods for collective I/O based on underlying parallel file system locking protocols

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
DataStager: scalable data staging services for petascale applications

Proceedings of the 18th ACM international symposium on High performance distributed computing
Adaptable, metadata rich IO methods for portable high performance IO

IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing

Performance of the community earth system model

Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
Efficient data IO for a Parallel Global Cloud Resolving Model

Environmental Modelling & Software
A new flexible coupler for earth system modeling developed for CCSM4 and CESM1

International Journal of High Performance Computing Applications
Computational performance of ultra-high-resolution capability in the Community Earth System Model

International Journal of High Performance Computing Applications
A modern solver interface to manage solution algorithms in the Community Earth System Model

International Journal of High Performance Computing Applications
Extending scalability of collective IO through nessie and staging

Proceedings of the sixth workshop on Parallel Data Storage

Quantified Score

Hi-index	0.00

Visualization

Abstract

We describe the design and implementation of an application-level parallel I/O (PIO) library for the reading and writing of distributed arrays to several common scientific data formats. PIO provides the flexibility to control the number of I/O tasks through data rearrangement to an I/O friendly decomposition. This flexibility enables reductions in per task memory usage and improvements in disk I/O performance versus a serial I/O approach. We illustrate the impact various features within PIO have on memory usage and disk I/O bandwidth on a Cray XT5 system.