Experiments with in-transit processing for data intensive grid workflows

Authors:
Virai Bhat;Manish Parashar;Scott Klasky
Affiliations:
Department of Electrical and Computer Engineering, Rutgers University, 94 Brett Road, Piscataway, NJ 08854-8058, U.S.A.;Department of Electrical and Computer Engineering, Rutgers University, 94 Brett Road, Piscataway, NJ 08854-8058, U.S.A.;Oak Ridge National Laboratory, P.O. Box 2008, TN, 37831, USA
Venue:
GRID '07 Proceedings of the 8th IEEE/ACM International Conference on Grid Computing
Year:
2007

Citing 11
Cited 2

Processing large-scale multi-dimensional data in parallel and distributed environments

Parallel Computing - Parallel data-intensive algorithms and applications
GridAnt: A Client-Controllable Grid Work.ow System

HICSS '04 Proceedings of the Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS'04) - Track 7 - Volume 7
SPHINX: A Fault-Tolerant System for Scheduling in Dynamic Grid Environments

IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
SPHINX: A Fault-Tolerant System for Scheduling in Dynamic Grid Environments

IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
The Globus Striped GridFTP Framework and Server

SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
High-Level Buffering for Hiding Periodic Output Cost in Scientific Simulations

IEEE Transactions on Parallel and Distributed Systems
The Center for Plasma Edge Simulation Workflow Requirements

ICDEW '06 Proceedings of the 22nd International Conference on Data Engineering Workshops
A Grid service broker for scheduling e-Science applications on global data Grids: Research Articles

Concurrency and Computation: Practice & Experience - Middleware for Grid Computing
Accord: a programming system for autonomic self-managing applications

Accord: a programming system for autonomic self-managing applications
Trickle: a userland bandwidth shaper for Unix-like systems

ATEC '05 Proceedings of the annual conference on USENIX Annual Technical Conference
A Self-Managing Wide-Area Data Streaming Service using Model-based Online Control

GRID '06 Proceedings of the 7th IEEE/ACM International Conference on Grid Computing

Investigating autonomic behaviours in grid-basedcomputational science applications

GMAC '09 Proceedings of the 6th international conference industry session on Grids meets autonomic computing
Self-adaptive architectures for autonomic computational science

SOAR'09 Proceedings of the First international conference on Self-organizing architectures

Quantified Score

Hi-index	0.00

Visualization

Abstract

Efficient and robust data streaming and in-transit data manipulations are critical requirements of emerging scientific and engineering application workflows, which are based on seamless interactions and coupling between geographically distributed application components. The overall goal of this research is to address these requirements and develop a data streaming and in-transit data manipulation service. In this paper, we experimentally investigate reactive management strategies for in-transit data manipulation, as well as cooperative end-to-end management for wide-area data-streaming and in-transit data manipulation for data-intensive scientific and engineering workflows.