Examining the Challenges of Scientific Workflows

Authors:
Yolanda Gil;Ewa Deelman;Mark Ellisman;Thomas Fahringer;Geoffrey Fox;Dennis Gannon;Carole Goble;Miron Livny;Luc Moreau;Jim Myers
Affiliations:
University of Southern California;University of Southern California;University of California, San Diego;University of Innsbruck;Indiana University;Indiana University;Manchester University;University of Wisconsin-Madison;University of Southampton;National Center for Supercomputing Applications
Venue:
Computer
Year:
2007

Citing 0
Cited 79

Efficient provenance storage

Proceedings of the 2008 ACM SIGMOD international conference on Management of data
Towards a System-Level Science Support

ICCS '08 Proceedings of the 8th international conference on Computational Science, Part I
International Workshop on Applications of Workflows in Computational Science (AWCS 08)

ICCS '08 Proceedings of the 8th international conference on Computational Science, Part III
A Framework for Interactive Parameter Sweep Applications

ICCS '08 Proceedings of the 8th international conference on Computational Science, Part III
Using Explicit Control Processes in Distributed Workflows to Gather Provenance

Provenance and Annotation of Data and Processes
The Open Provenance Model: An Overview

Provenance and Annotation of Data and Processes
A scientific workflow construction command line

Proceedings of the 14th international conference on Intelligent user interfaces
Editorial: Special section on workflow systems and applications in e-Science

Future Generation Computer Systems
Scientific workflow design for mere mortals

Future Generation Computer Systems
Heterogeneous composition of models of computation

Future Generation Computer Systems
Efficient provenance storage over nested data collections

Proceedings of the 12th International Conference on Extending Database Technology: Advances in Database Technology
Structured collaborative workflow design

Future Generation Computer Systems
What Makes Scientific Workflows Scientific?

SSDBM 2009 Proceedings of the 21st International Conference on Scientific and Statistical Database Management
Workflow matching using semantic metadata

Proceedings of the fifth international conference on Knowledge capture
Workflow enactment engine for WSRF-compliant services orchestration

GRID '08 Proceedings of the 2008 9th IEEE/ACM International Conference on Grid Computing
A Visual Interface for on-the-fly Biological Database Integration and Workflow Design Using VizBuilder

DILS '09 Proceedings of the 6th International Workshop on Data Integration in the Life Sciences
From data to knowledge to discoveries: Artificial intelligence and scientific workflows

Scientific Programming
Scientific Workflows: Business as Usual?

BPM '09 Proceedings of the 7th International Conference on Business Process Management
A framework for preservable geometry-centric artifacts

2009 SIAM/ACM Joint Conference on Geometric and Physical Modeling
Pipeline-centric provenance model

Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science
Workflow management in the grid era: A goal-driven approach based on process patterns

Multiagent and Grid Systems - New tendencies on agents and grid environments
Analysing scientific workflows with Computational Tree Logic

Cluster Computing
E-Biology Workflows with Calvin

WISE '09 Proceedings of the 10th International Conference on Web Information Systems Engineering
Fine-grained and efficient lineage querying of collection-based workflow provenance

Proceedings of the 13th International Conference on Extending Database Technology
An uncoordinated asynchronous checkpointing model for hierarchical scientific workflows

Journal of Computer and System Sciences
Scaling up workflow-based applications

Journal of Computer and System Sciences
Localising temporal constraints in scientific workflows

Journal of Computer and System Sciences
Efficiently supporting secure and reliable collaboration in scientific workflows

Journal of Computer and System Sciences
Streamflow Programming Model for Data Streaming in Scientific Workflows

CCGRID '10 Proceedings of the 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing
ParaTrac: a fine-grained profiler for data-intensive workflows

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
Trusted computing and provenance: better together

TAPP'10 Proceedings of the 2nd conference on Theory and practice of provenance
Towards a provenance framework for sub-image processing for astronomical data

CIKM '10 Proceedings of the 19th ACM international conference on Information and knowledge management
Taverna, reloaded

SSDBM'10 Proceedings of the 22nd international conference on Scientific and statistical database management
Learning deterministic finite automata from interleaved strings

ICGI'10 Proceedings of the 10th international colloquium conference on Grammatical inference: theoretical results and applications
A data lineage model for distributed sub-image processing

SAICSIT '10 Proceedings of the 2010 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists
The Foundations for Provenance on the Web

Foundations and Trends in Web Science
The Open Provenance Model core specification (v1.1)

Future Generation Computer Systems
Review: interactive knowledge capture in the new millennium: How the semantic web changed everything

The Knowledge Engineering Review
Velo: riding the knowledge management wave for simulation and modeling

Proceedings of the 4th International Workshop on Software Engineering for Computational Science and Engineering
RECYCLE: Learning looping workflows from annotated traces

ACM Transactions on Intelligent Systems and Technology (TIST)
On the long-term retention of geometry-centric digital engineering artifacts

Computer-Aided Design
Temporal dependency-based checkpoint selection for dynamic verification of temporal constraints in scientific workflow systems

ACM Transactions on Software Engineering and Methodology (TOSEM)
Agile Project Management: A Case Study of a Virtual Research Environment Development Project

Computer Supported Cooperative Work
PROPUB: towards a declarative approach for publishing customized, policy-aware provenance

SSDBM'11 Proceedings of the 23rd international conference on Scientific and statistical database management
Composer-Science: A semantic service based framework for workflow composition in e-Science projects

Information Sciences: an International Journal
A virtual sensor system for user-generated, real-time environmental data products

Environmental Modelling & Software
Scientific workflow reuse through conceptual workflows on the virtual imaging platform

Proceedings of the 6th workshop on Workflows in support of large-scale science
Workflow overhead analysis and optimizations

Proceedings of the 6th workshop on Workflows in support of large-scale science
Supporting dynamic parameter sweep in adaptive and user-steered workflow

Proceedings of the 6th workshop on Workflows in support of large-scale science
A new approach for publishing workflows: abstractions, standards, and linked data

Proceedings of the 6th workshop on Workflows in support of large-scale science
Making data analysis expertise broadly accessible through workflows

Proceedings of the 6th workshop on Workflows in support of large-scale science
Managing entire lifecycles of e-science applications in the gridspace2 virtual laboratory --- from motivation through idea to operable web-accessible environment built on top of PL-Grid e-infrastructure

Building a National Distributed e-Infrastructure - PL-Grid
Acquiring adaptation cases for scientific workflows

ICCBR'11 Proceedings of the 19th international conference on Case-Based Reasoning Research and Development
Component-based approach for programming and running scientific applications on grids and clouds

International Journal of High Performance Computing Applications
Parallel software architecture for experimental workflows in computational biology on clouds

PPAM'11 Proceedings of the 9th international conference on Parallel Processing and Applied Mathematics - Volume Part II
Eco-informatics modelling via semantic inference

Information Systems
Composing multiple variability artifacts to assemble coherent workflows

Software Quality Control
Nonintrusive collection and management of data provenance in scientific workflows

Concurrency and Computation: Practice & Experience
Declarative rules for inferring fine-grained data provenance from scientific workflow execution traces

IPAW'12 Proceedings of the 4th international conference on Provenance and Annotation of Data and Processes
Using domain-specific data to enhance scientific workflow steering queries

IPAW'12 Proceedings of the 4th international conference on Provenance and Annotation of Data and Processes
Management and storage of in situ oceanographic data: An ECM-based approach

Information Systems
WorMS- a framework to support workflows in M&S

Proceedings of the Winter Simulation Conference
Using workflows in M&S software

Proceedings of the Winter Simulation Conference
Optimizing execution time predictions of scientific workflow applications in the Grid through evolutionary programming

Future Generation Computer Systems
Adapting scientific workflow structures using multi-objective optimization strategies

ACM Transactions on Autonomous and Adaptive Systems (TAAS)
Reconstructing the software environment of an experiment with kameleon

Proceedings of the 5th ACM COMPUTE Conference: Intelligent & scalable system technologies
Scheduling parameter sweep workflow in the Grid based on resource competition

Future Generation Computer Systems
Performance evaluation of parallel strategies in public clouds: A study with phylogenomic workflows

Future Generation Computer Systems
Digital libraries for the preservation of research methods and associated artifacts

Proceedings of the 1st International Workshop on Digital Preservation of Research Methods and Artefacts
DynamicCloudSim: simulating heterogeneity in computational clouds

Proceedings of the 2nd ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
User-steering of HPC workflows: state-of-the-art and future directions

Proceedings of the 2nd ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
Time-bound analytic tasks on large datasets through dynamic configuration of workflows

WORKS '13 Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science
The demand for consistent web-based workflow editors

WORKS '13 Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science
Structured analysis of the ISI Atomic Pair Actions dataset using workflows

Pattern Recognition Letters
Computer-Assisted Scientific Workflow Design

Journal of Grid Computing
Research paper: A centralized tool for managing, archiving, and serving point-in-time data in ecological research laboratories

Environmental Modelling & Software
A graph distance based metric for data oriented workflow retrieval with variable time constraints

Expert Systems with Applications: An International Journal
Runtime Dynamic Structural Changes of Scientific Workflows in Clouds

UCC '13 Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing
Report from the second workshop on scalable workflow enactment engines and technology (SWEET'13)

ACM SIGMOD Record

Quantified Score

Hi-index	4.10

Visualization

Abstract

Workflows have emerged as a paradigm for representing and managing complex distributed computations and are used to accelerate the pace of scientific progress. A recent National Science Foundation workshop brought together domain, computer, and social scientists to discuss requirements of future scientific applications and the challenges they present to current workflow technologies.