Sun Grid Engine: Towards Creating a Compute Power Grid
CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
Distributed computing in practice: the Condor experience: Research Articles
Concurrency and Computation: Practice & Experience - Grid Performance
SSHFS: super easy file access over SSH
Linux Journal
Uncertainty estimation and prediction for interdisciplinary ocean dynamics
Journal of Computational Physics - Special issue: Uncertainty quantification in simulation science
A distributed file system for a wide-area high performance computing infrastructure
WORLDS'06 Proceedings of the 3rd conference on USENIX Workshop on Real, Large Distributed Systems - Volume 3
Journal of Field Robotics - Special Issue on Underwater Robotics
Personal adaptive clusters as containers for scientific jobs
Cluster Computing
Design and implementation of "many parallel task" hybrid subsurface model
Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers
Hi-index | 0.00 |
Error Subspace Statistical Estimation (ESSE), an uncertainty prediction and data assimilation methodology employed for real-time ocean forecasts, is based on a characterization and prediction of the largest uncertainties. This is carried out by evolving an error subspace of variable size. We use an ensemble of stochastic model simulations, initialized based on an estimate of the dominant initial uncertainties, to predict the error subspace of the model fields. The dominant error covariance (generated via an SVD of the ensemble-generated error covariance matrix) is used for data assimilation. The resulting ocean fields are provided as the input to acoustic modeling, allowing for the prediction and study of the spatiotemporal variations in acoustic propagation and their uncertainties. The ESSE procedure is a classic case of Many Task Computing: These codes are managed based on dynamic workflows for the: (i) perturbation of the initial mean state, (ii) subsequent ensemble of stochastic PE model runs, (iii) continuous generation of the covariance matrix, (iv) successive computations of the SVD of the ensemble spread until a convergence criterion is satisfied, and (v) data assimilation. Its ensemble nature makes it a many task data intensive application and its dynamic workflow gives it heterogeneity. Subsequent acoustics propagation modeling involves a very large ensemble of short-in-duration acoustics runs. We study the execution characteristics and challenges of a distributed ESSE workflow on a large dedicated cluster and the usability of enhancing this with runs on Amazon EC2 and the Teragrid and the I/O challenges faced.