Autonomic streaming pipeline for scientific workflows

  • Authors:
  • Rafael Tolosana-Calasanz;José A. Bañares;Omer F. Rana

  • Affiliations:
  • Instituto de Investigación en Ingeniería de Aragón (I3A), Department of Computer Science and Systems Engineering, Universidad de Zaragoza, Spain;Instituto de Investigación en Ingeniería de Aragón (I3A), Department of Computer Science and Systems Engineering, Universidad de Zaragoza, Spain;School of Computer Science, Cardiff University, Wales, U.K.

  • Venue:
  • Concurrency and Computation: Practice & Experience
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Interest in data streaming within scientific workflow has increased significantly over the recent years—mainly due to the emergence of data-driven applications. Such applications can include data streaming from sensors and data coupling between scientific simulations. To support resource management to enact such streaming-based workflow, autonomic computing techniques for transmission have been combined with in-transit processing, so that data elements may be processed in advance, enroute, prior to arrival at the destination. We propose the integration of an autonomic data streaming service (ADSS) with in-transit processing into a workflow specification. This integration may imply that the associated runtime resource allocation is dependent on environmental conditions and can change for different enactments of the same workflow. In our proposal, our workflow specifications are independent of the constraints imposed by the resource allocation. We express our solutions in terms of Reference nets. We also implement an ADSS utilizing a timed Reference net simulation for predicting future states of the ADSS. There are two advantages: the Reference net which implements the ADSS and the timed model are coincident, and second, token distribution obtained from the Petri net implementation can be utilized to better understand the demand for particular types of resources in the system. Copyright © 2011 John Wiley & Sons, Ltd.