A model for optimizing file access patterns using spatio-temporal parallelism

Authors:
Boonthanome Nouanesengsy;John Patchett;James Ahrens;Andrew Bauer;Aashish Chaudhary;Ross Miller;Berk Geveci;Galen M. Shipman;Dean N. Williams
Affiliations:
Los Alamos National Laboratory;Los Alamos National Laboratory;Los Alamos National Laboratory;Kitware Inc.;Kitware Inc.;Oak Ridge National Laboratory;Kitware Inc.;Oak Ridge National Laboratory;Lawrence Livermore National Laboratory
Venue:
UltraVis '13 Proceedings of the 8th International Workshop on Ultrascale Visualization
Year:
2013

Citing 9
Cited 0

A Parallel Visualization Pipeline for Terascale Earthquake Simulations

Proceedings of the 2004 ACM/IEEE conference on Supercomputing
A study of I/O methods for parallel visualization of large-scale data

Parallel Computing - Parallel graphics and visualization
Time Dependent Processing in a Parallel Pipeline Architecture

IEEE Transactions on Visualization and Computer Graphics
Extreme Scaling of Production Visualization Software on Diverse Architectures

IEEE Computer Graphics and Applications
Accelerating data-intensive science with Gordon and Dash

Proceedings of the 2010 TeraGrid Conference
VisIO: Enabling Interactive Visualization of Ultra-Scale, Time Series Data via High-Bandwidth Distributed I/O Systems

IPDPS '11 Proceedings of the 2011 IEEE International Parallel & Distributed Processing Symposium
Toward a General I/O Layer for Parallel-Visualization Applications

IEEE Computer Graphics and Applications
Parallel high-resolution climate data analysis using swift

Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers
Parallel in situ coupling of simulation with a fully featured visualization system

EG PGV'11 Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization

Quantified Score

Hi-index	0.00

Visualization

Abstract

For many years now, I/O read time has been recognized as the primary bottleneck for parallel visualization and analysis of large-scale data. In this paper, we introduce a model that can estimate the read time for a file stored in a parallel filesystem when given the file access pattern. Read times ultimately depend on how the file is stored and the access pattern used to read the file. The file access pattern will be dictated by the type of parallel decomposition used. We employ spatio-temporal parallelism, which combines both spatial and temporal parallelism, to provide greater flexibility to possible file access patterns. Using our model, we were able to configure the spatio-temporal parallelism to design optimized read access patterns that resulted in a speedup factor of approximately 400 over traditional file access patterns.