On the black art of designing computational workflows

Authors:
Yolanda Gil;Pedro A. González-Calero;Ewa Deelman
Affiliations:
USC Information Sciences Institute, Marina Del Rey, CA;Universidad Complutense de Madrid, Madrid, Spain;USC Information Sciences Institute, Marina Del Rey, CA
Venue:
Proceedings of the 2nd workshop on Workflows in support of large-scale science
Year:
2007

Citing 14
Cited 3

The Pegasus portal: web based grid computing

Proceedings of the 2005 ACM symposium on Applied computing
Editor's Introduction: Adapting Agile Approaches to Your Project Needs

IEEE Software
A Comparison of Two Methods for Building Astronomical Image Mosaics on a Grid

ICPPW '05 Proceedings of the 2005 International Conference on Parallel Processing Workshops
Software Reuse Research: Status and Future

IEEE Transactions on Software Engineering
Simplifying construction of complex workflows for non-expert users of the Southern California Earthquake Center Community Modeling Environment

ACM SIGMOD Record
Taverna: lessons in creating a workflow environment for the life sciences: Research Articles

Concurrency and Computation: Practice & Experience - Workflow in Grid Systems
Scientific workflow management and the Kepler system: Research Articles

Concurrency and Computation: Practice & Experience - Workflow in Grid Systems
Task scheduling strategies for workflow-based applications in grids

CCGRID '05 Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2 - Volume 02
New methods in software product line practice

Communications of the ACM - Software product line
Managing Large-Scale Scientific Workflows in Distributed Environments: Experiences and Challenges

E-SCIENCE '06 Proceedings of the Second IEEE International Conference on e-Science and Grid Computing
Pegasus: A framework for mapping complex scientific workflows onto distributed systems

Scientific Programming
Special Issue: The First Provenance Challenge

Concurrency and Computation: Practice & Experience - The First Provenance Challenge
Wings for Pegasus: creating large-scale scientific applications using semantic representations of computational workflows

IAAI'07 Proceedings of the 19th national conference on Innovative applications of artificial intelligence - Volume 2
Semantic metadata generation for large scientific workflows

ISWC'06 Proceedings of the 5th international conference on The Semantic Web

Semantic templates for case-based reasoning systems

The Knowledge Engineering Review
Case studies in storage access by loosely coupled petascale applications

Proceedings of the 4th Annual Workshop on Petascale Data Storage
Towards the conception of scientific workflows for in silico experiments in software engineering

Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement

Quantified Score

Hi-index	0.00

Visualization

Abstract

Computational workflows have recently emerged as an effective paradigm to manage large-scale distributed scientific computations. Workflow systems can automate many execution-level details and provide assistance in composing and validating workflows. However, there is still a significant effort involved in creating these workflows since they often represent collaborative and exploratory science experiments. Therefore, current practice is effective in producing results but not cost-effective for widespread adoption. Drawing from our previous research in computational workflows across scientific disciplines, this paper analyzes the tasks and overall process for designing these workflows. We discuss software engineering methodologies and their relevance to creating workflows as a unique kind of software artifact. We also discuss our ongoing work to make workflow applications more cost effective and lower the barriers for widespread adoption of workflow technologies.