A patch-based data reorganization method for coupling large-scale simulations and parallel visualization

Authors:
Li Xiao;Zhiwei Ai;Xiaolin Cao
Affiliations:
Institute of Applied Physics and Computational Mathematics, Beijing, China;Institute of Applied Physics and Computational Mathematics, Beijing, China;Institute of Applied Physics and Computational Mathematics, Beijing, China
Venue:
Transactions on Edutainment IX
Year:
2013

Citing 8
Cited 0

A study of I/O methods for parallel visualization of large-scale data

Parallel Computing - Parallel graphics and visualization
ZOID: I/O-forwarding infrastructure for petascale architectures

Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming
An efficient format for nearly constant-time access to arbitrary time intervals in large trace files

Scientific Programming - Large-Scale Programming Tools and Environments
I/O performance challenges at leadership scale

Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
In Situ Visualization at Extreme Scale: Challenges and Opportunities

IEEE Computer Graphics and Applications
Extreme Scaling of Production Visualization Software on Diverse Architectures

IEEE Computer Graphics and Applications
BlobSeer: Next-generation data management for large scale infrastructures

Journal of Parallel and Distributed Computing
JASMIN: a parallel software infrastructure for scientific computing

Frontiers of Computer Science in China

Quantified Score

Hi-index	0.00

Visualization

Abstract

The scale of some datasets generated by simulations on tens of thousands of cores are gigabyte or larger per output step. It is imperative that efficient coupling of these simulations and parallel visualization. A patch-based data reorganization method was presented for this coupling through a parallel file system. Based on the method, simulation data sets in application codes are reorganized by patch and written into many files in parallel. These datasets in these files can be read directly by visualization software with low I/O overheads. For two real simulations on above 30000 cores, large-scale datasets have been generated and visualized efficiently.