NEST: a network simulation and prototyping testbed
Communications of the ACM - Special issue on simulation
An overview of workflow management: from process modeling to workflow automation infrastructure
Distributed and Parallel Databases - Special issue on software support for work flow management
Simulation modeling within workflow technology
WSC '95 Proceedings of the 27th conference on Winter simulation
IEEE Transactions on Parallel and Distributed Systems
MCI—multilanguage distributed co-simulation tool
DIPES '98 Proceedings of the IFIP WG10.3/WG10.5 international workshop on Distributed and parallel embedded systems
TriGS/sub flow/: Active object-oriented workflow management
HICSS '95 Proceedings of the 28th Hawaii International Conference on System Sciences
Using Simulation to Facilitate Effective Workflow Adaptation
SS '02 Proceedings of the 35th Annual Simulation Symposium
Semantic E-Workflow Composition
Journal of Intelligent Information Systems
Scheduling of scientific workflows in the ASKALON grid environment
ACM SIGMOD Record
The SARA system for computer architecture design and modelling
ACM SIGSIM Simulation Digest
Scientific workflow management and the Kepler system: Research Articles
Concurrency and Computation: Practice & Experience - Workflow in Grid Systems
Programming scientific and distributed workflow with Triana services: Research Articles
Concurrency and Computation: Practice & Experience - Workflow in Grid Systems
E-SCIENCE '06 Proceedings of the Second IEEE International Conference on e-Science and Grid Computing
Pegasus: A framework for mapping complex scientific workflows onto distributed systems
Scientific Programming
A Dynamic Critical Path Algorithm for Scheduling Scientific Workflow Applications on Global Grids
E-SCIENCE '07 Proceedings of the Third IEEE International Conference on e-Science and Grid Computing
Web process and workflow path mining using the Multimethod approach
International Journal of Business Intelligence and Data Mining
Mapping pipeline skeletons onto heterogeneous platforms
Journal of Parallel and Distributed Computing
Efficient pipeline configuration in distributed heterogeneous computing environments
Proceedings of the twenty-seventh ACM symposium on Principles of distributed computing
SODA: An Optimizing Scheduler for Large-Scale Stream-Based Distributed Computer Systems
Middleware '08 Proceedings of the ACM/IFIP/USENIX 9th International Middleware Conference
Supporting Distributed Application Workflows in Heterogeneous Computing Environments
ICPADS '08 Proceedings of the 2008 14th IEEE International Conference on Parallel and Distributed Systems
SpringSim '09 Proceedings of the 2009 Spring Simulation Multiconference
Optimizing End-to-end Performance of Distributed Applications with Linear Computing Pipelines
ICPADS '09 Proceedings of the 2009 15th International Conference on Parallel and Distributed Systems
Optimizing distributed computing workflows in heterogeneous network environments
ICDCN'10 Proceedings of the 11th international conference on Distributed computing and networking
Proceedings of the 44th Annual Simulation Symposium
Hi-index | 0.00 |
Next-generation computation- and network-intensive collaborative applications in various science, engineering, and e-commerce fields feature large-scale computing workflows of complex structures. Efficient algorithms are needed for task scheduling, module deployment, and service provisioning to support the execution of such distributed workflows in heterogeneous network environments and optimize their end-to-end performance for fast system response or smooth data flow. However, deploying large-scale distributed applications in real network environments is extremely challenging due to the inherent dynamics in the reliability, availability, accessibility, and capacity of massively distributed system resources, which are typically shared among a broad community of users over the Internet or dedicated connections. We propose a simulation system to study the execution dynamics of distributed computing workflows and evaluate the network performance of workflow scheduling or mapping algorithms before actual deployment and experimentation. The proposed simulation system visually illustrates the dynamic execution process of workflows in network environments by simulating module execution on computer nodes and data transfer over network links in a completely distributed and parallel manner. Furthermore, the simulation system takes background traffic and workload into consideration to achieve a high level of simulation accuracy for distributed applications deployed in shared production network environments. We implement the simulation system using multi-threaded programming and conduct extensive testings on various mapping schemes using a large number of simulated workflows and networks. The simulation-based performance measurements are quantitatively confirmed by both the experimental observations collected in real networks and the theoretical results obtained by rigorous performance analysis based on well-defined mathematical models.