Supporting dynamic parameter sweep in adaptive and user-steered workflow
Proceedings of the 6th workshop on Workflows in support of large-scale science
A task routing approach to large-scale scheduling
Future Generation Computer Systems
Scheduling parameter sweep workflow in the Grid based on resource competition
Future Generation Computer Systems
Approaches to Distributed Execution of Scientific Workflows in Kepler
Fundamenta Informaticae - Scalable Workflow Enactment Engines and Technology
| Hi-index | 0.00 |
Robust scientific methods require the exploration of the parameter space of a system (some of which can be run in parallel on distributed resources), and may involve complete state space exploration, experimental design, or numerical optimization techniques. Many-Task Computing (MTC) provides a framework for performing robust design, because it supports the execution of a large number of otherwise independent processes. Further, scientific workflow engines facilitate the specification and execution of complex software pipelines, such as those found in real science and engineering design problems. However, most existing workflow engines do not support a wide range of experimentation techniques, nor do they support a large number of independent tasks. In this paper, we discuss Nimrod/K—a set of add in components and a new run time machine for a general workflow engine, Kepler. Nimrod/K provides an execution architecture based on the tagged dataflow concepts, developed in 1980s for highly parallel machines. This is embodied in a new Kepler "Director” that supports many-task computing by orchestrating execution of tasks on on clusters, Grids, and Clouds. Further, Nimrod/K provides a set of "Actors” that facilitate the various modes of parameter exploration discussed above. We demonstrate the power of Nimrod/K to solve real problems in cardiac science.