Gigabit networking
An approach for pipelining nested collections in scientific workflows
ACM SIGMOD Record
Taverna: lessons in creating a workflow environment for the life sciences: Research Articles
Concurrency and Computation: Practice & Experience - Workflow in Grid Systems
Parallelizing XML data-streaming workflows via MapReduce
Journal of Computer and System Sciences
SSDBM'10 Proceedings of the 22nd international conference on Scientific and statistical database management
Adaptive rate stream processing for smart grid applications on clouds
Proceedings of the 2nd international workshop on Scientific cloud computing
Towards Reliable, Performant Workflows for Streaming-Applications on Cloud Platforms
CCGRID '11 Proceedings of the 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing
Autonomic streaming pipeline for scientific workflows
Concurrency and Computation: Practice & Experience
Enforcing QoS in scientific workflow systems enacted over Cloud infrastructures
Journal of Computer and System Sciences
Revenue-Based resource management on shared clouds for heterogenous bursty data streams
GECON'12 Proceedings of the 9th international conference on Economics of Grids, Clouds, Systems, and Services
Exploiting application dynamism and cloud elasticity for continuous dataflows
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Cloud Based Big Data Analytics for Smart Future Cities
UCC '13 Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing
| Hi-index | 0.00 |
The increasing deployment of sensor network infrastructures has led to large volumes of data becoming available, leading to new challenges in storing, processing and transmitting such data. This is especially true when data from multiple sensors is pre-processed prior to delivery to users. Where such data is processed in-transit (i.e. from data capture to delivery to a user) over a shared distributed computing infrastructure, it is necessary to provide some Quality of Service (QoS) guarantees to each user. We propose an architecture for supporting QoS for multiple concurrent scientific workflow data streams being processed (prior to delivery to a user) over a shared infrastructure. We consider such an infrastructure to be composed of a number of nodes, each of which has multiple processing units and data buffers. We utilize the ``token bucket" model for regulating, on a per workflow stream basis, the data injection rate into such a node. We subsequently demonstrate how a streaming pipeline, with intermediate data size variation (inflation/deflation), can be supported and managed using a dynamic control strategy at each node. Such a strategy supports end-to-end QoS with variations in data size between the various nodes involved in the workflow enactment process.