Component-based stream processing "in the cloud"

Authors:
Paul N. Martinaitis;Craig J. Patten;Andrew L. Wendelborn
Affiliations:
University of Adelaide, SA, Australia;University of Adelaide, SA, Australia;University of Adelaide, SA, Australia
Venue:
Proceedings of the 2009 Workshop on Component-Based High Performance Computing
Year:
2009

Citing 5
Cited 4

Sun Grid Engine: Towards Creating a Compute Power Grid

CCGRID '01 Proceedings of the 1st International Symposium on Cluster Computing and the Grid
A grid service broker for scheduling distributed data-oriented applications on global grids

MGC '04 Proceedings of the 2nd workshop on Middleware for grid computing
The FRACTAL component model and its support in Java: Experiences with Auto-adaptive and Reconfigurable Systems

Software—Practice & Experience
Binary Data Transfer Performance over High-Latency Networks Using Web Service Attachments

E-SCIENCE '07 Proceedings of the Third IEEE International Conference on e-Science and Grid Computing
Stream-components: component based stream computation on the grid

AusGrid '09 Proceedings of the Seventh Australasian Symposium on Grid Computing and e-Research - Volume 99

Representing Eager Evaluation in a Demand Driven Model of Streams on Cloud Infrastructure

CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
Towards component-based software engineering of cloud applications

Proceedings of the WICSA/ECSA 2012 Companion Volume
Parameterised architectural patterns for providing cloud service fault tolerance with accurate costings

Proceedings of the 16th International ACM Sigsoft symposium on Component-based software engineering
Developing a Model Driven Approach for engineering applications based on mOSAIC

Cluster Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

In earlier work, we reported on modeling of stream processing in terms of distributed components (as exemplified in the EU CoreGrid project ProActive), showing how a stream processing system can be built from components in composition, with dynamic reconfiguration and distributed management of the streams. In this paper, we introduce the Web Service Stream Deployer (WSSD), for the remote establishment and deployment of streams across widely distributed resources, allowing a user to set-up, control and reconfigure a stream remotely and dynamically. We demonstrate this concept with streams on a cloud testbed, using Nimbus cloud infrastructure at the University of Chicago. In particular, we show that our web-services based WSSD exhibits minimal adverse latency effect when used over an intercontinental network to manipulate, from our client in Australia, a cloud-based stream in the USA; we believe that this represents a useful mode of remote interaction with cloud-based applications. We outline some ideas in scheduling of cloud-based streams; we present a model of interaction with Gridbus, a widely used framework for exploring different scheduling algorithms (such as economic scheduling), whereby scheduling criteria expressed in Gridbus can be used to automatically deploy stream components.