Run-time creation of the turbulent channel flow database by an HPC simulation using MPI-DB

Authors:
Jason Graham;Edward Givelberg;Kalin Kanov
Affiliations:
Johns Hopkins University, Baltimore, MD;Johns Hopkins University, Baltimore, MD;Johns Hopkins University, Baltimore, MD
Venue:
Proceedings of the 20th European MPI Users' Group Meeting
Year:
2013

Citing 10
Cited 0

A query language for multidimensional arrays: design, implementation, and optimization techniques

SIGMOD '96 Proceedings of the 1996 ACM SIGMOD international conference on Management of data
The multidimensional database system RasDaMan

SIGMOD '98 Proceedings of the 1998 ACM SIGMOD international conference on Management of data
Data Management: NetCDF: an Interface for Scientific Data Access

IEEE Computer Graphics and Applications
Query processing techniques for arrays

The VLDB Journal — The International Journal on Very Large Data Bases
UDT: UDP-based data transfer for high-speed wide area networks

Computer Networks: The International Journal of Computer and Telecommunications Networking
Dynamo: amazon's highly available key-value store

Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles
Data exploration of turbulence simulations using a database cluster

Proceedings of the 2007 ACM/IEEE conference on Supercomputing
Bigtable: A Distributed Storage System for Structured Data

ACM Transactions on Computer Systems (TOCS)
Overview of sciDB: large scale array storage, processing and analysis

Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
An architecture for a data-intensive computer

Proceedings of the first international workshop on Network-aware data management

Quantified Score

Hi-index	0.00

Visualization

Abstract

We demonstrate a method based on MPI client-server implementation for storing the output of computations directly into the database. Our method automates the previously used inefficient ingestion process which required development of special tools for each simulation. In large-scale channel flow simulation experiments we were able to ingest the output data sets in real time, without delaying the simulation process. This was accomplished by building a Fortran interface to the MPI-DB software library and using it within the simulation code. The channel flow simulation data set will be exposed for analysis to researchers using the JHU Public Turbulence Database [7].