User-steering of HPC workflows: state-of-the-art and future directions

Authors:
Marta Mattoso;Kary Ocaña;Felipe Horta;Jonas Dias;Eduardo Ogasawara;Vitor Silva;Daniel de Oliveira;Flavio Costa;Igor Araújo
Affiliations:
Federal University of Rio de Janeiro, Rio de Janeiro, Brazil;Federal University of Rio de Janeiro, Rio de Janeiro, Brazil;Federal University of Rio de Janeiro, Rio de Janeiro, Brazil;Federal University of Rio de Janeiro, Rio de Janeiro, Brazil;Federal Center for Technological Education, Rio de Janeiro, Brazil;Federal University of Rio de Janeiro, Rio de Janeiro, Brazil;Fluminense Federal University, Niterói, Brazil;Federal University of Rio de Janeiro, Rio de Janeiro, Brazil;Federal University of Rio de Janeiro, Rio de Janeiro, Brazil
Venue:
Proceedings of the 2nd ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
Year:
2013

Citing 29
Cited 0

The grid: blueprint for a new computing infrastructure

The grid: blueprint for a new computing infrastructure
Enabling ScientificWorkflow Reuse through Structured Composition of Dataflow and Control-Flow

ICDEW '06 Proceedings of the 22nd International Conference on Data Engineering Workshops
VisTrails: visualization meets data management

Proceedings of the 2006 ACM SIGMOD international conference on Management of data
K-WfGrid Distributed Monitoring and Performance Analysis Services for Workflows in the Grid

E-SCIENCE '06 Proceedings of the Second IEEE International Conference on e-Science and Grid Computing
Workflow adaptation as an autonomic computing problem

Proceedings of the 2nd workshop on Workflows in support of large-scale science
Examining the Challenges of Scientific Workflows

Computer
Provenance Services for Distributed Workflows

CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Data transfers in the grid: workload analysis of globus GridFTP

DADC '08 Proceedings of the 2008 international workshop on Data-aware distributed computing
Provenance for Computational Tasks: A Survey

Computing in Science and Engineering
A break in the clouds: towards a cloud definition

ACM SIGCOMM Computer Communication Review
Workflows and e-Science: An overview of workflow system features and capabilities

Future Generation Computer Systems
Exploring many task computing in scientific workflows

Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers
Managing scientific data

Communications of the ACM
In Situ Visualization for Large-Scale Combustion Simulations

IEEE Computer Graphics and Applications
SciCumulus: A Lightweight Cloud Middleware to Explore Many Task Computing Paradigm in Scientific Workflows

CLOUD '10 Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing
Taverna, reloaded

SSDBM'10 Proceedings of the 22nd international conference on Scientific and statistical database management
Managing scientific data: lessons, challenges, and opportunities

Proceedings of the 2011 ACM SIGMOD International Conference on Management of data
SciPhy: a cloud-based workflow for phylogenetic analysis of drug targets in protozoan genomes

BSB'11 Proceedings of the 6th Brazilian conference on Advances in bioinformatics and computational biology
Supporting dynamic parameter sweep in adaptive and user-steered workflow

Proceedings of the 6th workshop on Workflows in support of large-scale science
Optimizing bioinformatics workflows for data analysis using cloud management techniques

Proceedings of the 6th workshop on Workflows in support of large-scale science
Failure prediction and localization in large scientific workflows

Proceedings of the 6th workshop on Workflows in support of large-scale science
Optimizing Phylogenetic Analysis Using SciHmm Cloud-based Scientific Workflow

ESCIENCE '11 Proceedings of the 2011 IEEE Seventh International Conference on eScience
Swift: A language for distributed parallel scripting

Parallel Computing
A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds

Journal of Grid Computing
Parallel in situ coupling of simulation with a fully featured visualization system

EG PGV'11 Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization
Enabling re-executions of parallel scientific workflows using runtime provenance data

IPAW'12 Proceedings of the 4th international conference on Provenance and Annotation of Data and Processes
Capturing and querying workflow runtime provenance with PROV: a practical approach

Proceedings of the Joint EDBT/ICDT 2013 Workshops
Discovering drug targets for neglected diseases using a pharmacophylogenomic cloud workflow

E-SCIENCE '12 Proceedings of the 2012 IEEE 8th International Conference on E-Science (e-Science)
Abstract: Using Provenance to Visualize Data from Large-Scale Experiments

SCC '12 Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

In 2006 a group of leading researchers was gathered to discuss several challenges to scientific workflow supporting technologies and many of which still remain open challenges, such as the steering of workflows by users. Due to big data and long lasting workflows, many users demand steering features such as real-time monitoring, analysis and specially execution interference. The workflow execution should respond dynamically to such interference in the execution, to support the experimentation process in high performance computing. This paper revisits the issues in the user steering and dynamic workflows, presenting the state-of-the-art in it, and the open challenges. Our goal is to discuss research issues related to scientists' steering and present some ideas on how these demands may be supported in current scientific workflow technologies.