Terascale data organization for discovering multivariate climatic trends

Authors:
Wesley Kendall;Markus Glatter;Jian Huang;Tom Peterka;Robert Latham;Robert Ross
Affiliations:
The University of Tennessee, Knoxville, TN;The University of Tennessee, Knoxville, TN;The University of Tennessee, Knoxville, TN;Argonne National Laboratory, Argonne, IL;Argonne National Laboratory, Argonne, IL;Argonne National Laboratory, Argonne, IL
Venue:
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
Year:
2009

Citing 19
Cited 6

Marching cubes: A high resolution 3D surface construction algorithm

SIGGRAPH '87 Proceedings of the 14th annual conference on Computer graphics and interactive techniques
A practical evaluation of popular volume rendering algorithms

VVS '00 Proceedings of the 2000 IEEE symposium on Volume visualization
Tracking and Visualizing Turbulent 3D Features

IEEE Transactions on Visualization and Computer Graphics
Data Management: NetCDF: an Interface for Scientific Data Access

IEEE Computer Graphics and Applications
Data Sieving and Collective I/O in ROMIO

FRONTIERS '99 Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation
A Parallel Visualization Pipeline for Terascale Earthquake Simulations

Proceedings of the 2004 ACM/IEEE conference on Supercomputing
Parallel netCDF: A High-Performance Scientific I/O Interface

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Visualizing Very Large-Scale Earthquake Simulations

Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Intelligent Feature Extraction and Tracking for Visualizing Large-Scale 4D Flow Simulations

SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
Multicollective I/O: A technique for exploiting inter-file access patterns

ACM Transactions on Storage (TOS)
Scalable Data Servers for Large Multivariate Volume Visualization

IEEE Transactions on Visualization and Computer Graphics
Detecting distributed scans using high-performance query-driven visualization

Proceedings of the 2006 ACM/IEEE conference on Supercomputing
Variable Interactions in Query-Driven Visualization

IEEE Transactions on Visualization and Computer Graphics
High performance multivariate visual data exploration for extremely large data

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Query-Driven Visualization of Time-Varying Adaptive Mesh Refinement Data

IEEE Transactions on Visualization and Computer Graphics
Visualizing Temporal Patterns in Large Multivariate Data using Modified Globbing

IEEE Transactions on Visualization and Computer Graphics
Adaptable, metadata rich IO methods for portable high performance IO

IPDPS '09 Proceedings of the 2009 IEEE International Symposium on Parallel&Distributed Processing
I/O strategies for parallel rendering of large time-varying volume data

EG PGV'04 Proceedings of the 5th Eurographics conference on Parallel Graphics and Visualization
Parallel volume rendering on the IBM Blue Gene/P

EG PGV'08 Proceedings of the 8th Eurographics conference on Parallel Graphics and Visualization

Dynamic tiled map services: supporting query-based visualization of large-scale raster geospatial data

Proceedings of the 1st International Conference and Exhibition on Computing for Geospatial Research & Application
Supporting web-based visual exploration of large-scale raster geospatial data using binned min-max Quadtree

SSDBM'10 Proceedings of the 22nd international conference on Scientific and statistical database management
Simplified parallel domain traversal

Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
Efficient I/O for parallel visualization

EG PGV'11 Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization
Parallel particle advection and FTLE computation for time-varying flow fields

SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Visualization for the Physical Sciences

Computer Graphics Forum

Quantified Score

Hi-index	0.00

Visualization

Abstract

Current visualization tools lack the ability to perform full-range spatial and temporal analysis on terascale scientific datasets. Two key reasons exist for this shortcoming: I/O and postprocessing on these datasets are being performed in suboptimal manners, and the subsequent data extraction and analysis routines have not been studied in depth at large scales. We resolved these issues through advanced I/O techniques and improvements to current query-driven visualization methods. We show the efficiency of our approach by analyzing over a terabyte of multivariate satellite data and addressing two key issues in climate science: time-lag analysis and drought assessment. Our methods allowed us to reduce the end-to-end execution times on these problems to one minute on a Cray XT4 machine.