Communications of the ACM
Fortran M: a language for modular parallel programming
Journal of Parallel and Distributed Computing
Server-directed collective I/O in Panda
Supercomputing '95 Proceedings of the 1995 ACM/IEEE conference on Supercomputing
Disk-directed I/O for MIMD multiprocessors
ACM Transactions on Computer Systems (TOCS)
Toward a Common Component Architecture for High-Performance Scientific Computing
HPDC '99 Proceedings of the 8th IEEE International Symposium on High Performance Distributed Computing
A lightweight Java taskspaces framework for scientific computing on computational grids
Proceedings of the 2003 ACM symposium on Applied computing
International Journal of High Performance Computing Applications
International Journal of High Performance Computing Applications
Opus: A Coordination Language for Multidisciplinary Applications
Scientific Programming
DART: a substrate for high speed asynchronous data IO
HPDC '08 Proceedings of the 17th international symposium on High performance distributed computing
DataStager: scalable data staging services for petascale applications
Proceedings of the 18th ACM international symposium on High performance distributed computing
Enabling high-speed asynchronous data extraction and transfer using DART
Concurrency and Computation: Practice & Experience
Enabling efficient and flexible coupling of parallel scientific applications
IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
HiPC'06 Proceedings of the 13th international conference on High Performance Computing
A dynamic geometry-based shared space interaction framework for parallel scientific applications
HiPC'04 Proceedings of the 11th international conference on High Performance Computing
Making a case for distributed file systems at Exascale
Proceedings of the third international workshop on Large-scale system and application performance
Just in time: adding value to the IO pipelines of high performance applications with JITStaging
Proceedings of the 20th international symposium on High performance distributed computing
Examples of in transit visualization
Proceedings of the 2nd international workshop on Petascal data analytics: challenges and opportunities
High end scientific codes with computational I/O pipelines: improving their end-to-end performance
Proceedings of the 2nd international workshop on Petascal data analytics: challenges and opportunities
Addressing the petascale data challenge using in-situ analytics
Proceedings of the 2nd international workshop on Petascal data analytics: challenges and opportunities
Extending scalability of collective IO through nessie and staging
Proceedings of the sixth workshop on Parallel Data Storage
In-situ I/O processing: a case for location flexibility
Proceedings of the sixth workshop on Parallel Data Storage
Exploring distributed hash tables in HighEnd computing
ACM SIGMETRICS Performance Evaluation Review
Combining in-situ and in-transit processing to enable extreme-scale scientific analysis
SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Extending MPI to better support multi-application interaction
EuroMPI'12 Proceedings of the 19th European conference on Recent Advances in the Message Passing Interface
Spatiotemporal data mining in the era of big spatial data: algorithms and applications
Proceedings of the 1st ACM SIGSPATIAL International Workshop on Analytics for Big Geospatial Data
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Exploring power behaviors and trade-offs of in-situ data analytics
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Hi-index | 0.00 |
Emerging high-performance distributed computing environments are enabling new end-to-end formulations in science and engineering that involve multiple interacting processes and data-intensive application workflows. For example, current fusion simulation efforts are exploring coupled models and codes that simultaneously simulate separate application processes, such as the core and the edge turbulence, and run on different high performance computing resources. These components need to interact, at runtime, with each other and with services for data monitoring, data analysis and visualization, and data archiving. As a result, they require efficient support for dynamic and flexible couplings and interactions, which remains a challenge. This paper presents DataSpaces, a flexible interaction and coordination substrate that addresses this challenge. DataSpaces essentially implements a semantically specialized virtual shared space abstraction that can be associatively accessed by all components and services in the application workflow. It enables live data to be extracted from running simulation components, indexes this data online, and then allows it to be monitored, queried and accessed by other components and services via the space using semantically meaningful operators. The underlying data transport is asynchronous, low-overhead and largely memory-to-memory. The design, implementation, and experimental evaluation of DataSpaces using a coupled fusion simulation workflow is presented.