Decentralized orchestration of data-centric workflows in Cloud environments

  • Authors:

  • Bahman Javadi;Martin Tomko;Richard O. Sinnott

  • Affiliations:

  • School of Computing, Engineering and Mathematics, University of Western Sydney, Australia;Faculty of Architecture, Building and Planning, The University of Melbourne, Australia;Department of Computing and Information Systems, The University of Melbourne, Australia

  • Venue:
  • Future Generation Computer Systems
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Data-centric and service-oriented workflows are commonly used in scientific research to enable the composition and execution of complex analysis on distributed resources. Although there are a plethora of orchestration frameworks to implement workflows, most of them are unsuitable for executing (enacting) data-centric workflows since they are based on a centralized orchestration engine which can be a bottleneck when handling large data volumes. In this paper, we propose a flexible and lightweight workflow framework based on the Object Modeling System (OMS). Moreover, we take advantage of the OMS architecture to deploy and execute data-centric workflows in a decentralized manner across multiple distinct Cloud resources, avoiding limitations of all data passing through a centralized engine. The proposed framework is implemented in the context of the Australian Urban Research Infrastructure Network (AURIN) project which is an initiative aiming to develop an e-Infrastructure supporting research in the urban and built environment domains. Performance evaluation results using spatial data-centric workflows show that we can reduce 20% of the workflow execution time when using Cloud resources in the same network domain.