Condor-G: A Computation Management Agent for Multi-Institutional Grids
Cluster Computing
OpenMP: An Industry-Standard API for Shared-Memory Programming
IEEE Computational Science & Engineering
The Philosophy of TeraGrid: Building an Open, Extensible, Distributed TeraScale Facility
CCGRID '02 Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid
Workflow management in GriPhyN
Grid resource management
E-SCIENCE '06 Proceedings of the Second IEEE International Conference on e-Science and Grid Computing
K-WfGrid Distributed Monitoring and Performance Analysis Services for Workflows in the Grid
E-SCIENCE '06 Proceedings of the Second IEEE International Conference on e-Science and Grid Computing
Online Analysis and Runtime Steering of Dynamic Workflows in the ASKALON Grid Environment
CCGRID '07 Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid
File system design for an NFS file server appliance
WTEC'94 Proceedings of the USENIX Winter 1994 Technical Conference on USENIX Winter 1994 Technical Conference
PVFS: a parallel file system for linux clusters
ALS'00 Proceedings of the 4th annual Linux Showcase & Conference - Volume 4
Overhead Analysis of Scientific Workflows in Grid Environments
IEEE Transactions on Parallel and Distributed Systems
Workflow task clustering for best effort systems with Pegasus
Proceedings of the 15th ACM Mardi Gras conference: From lightweight mash-ups to lambda grids: Understanding the spectrum of distributed computing requirements, applications, tools, infrastructures, interoperability, and the incremental adoption of key capabilities
Grid Computing: Achievements and Prospects
Grid Computing: Achievements and Prospects
Effective performance measurement and analysis of multithreaded applications
Proceedings of the 14th ACM SIGPLAN symposium on Principles and practice of parallel programming
Data placement for scientific applications in distributed environments
GRID '07 Proceedings of the 8th IEEE/ACM International Conference on Grid Computing
An integrated framework for performance-based optimization of scientific workflows
Proceedings of the 18th ACM international symposium on High performance distributed computing
A performance study of grid workflow engines
GRID '08 Proceedings of the 2008 9th IEEE/ACM International Conference on Grid Computing
Experiences with resource provisioning for scientific workflows using Corral
Scientific Programming
ParaTrac: a fine-grained profiler for data-intensive workflows
Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
DAGwoman: enabling DAGMan-like workflows on non-Condor platforms
Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
Oozie: towards a scalable workflow management system for Hadoop
Proceedings of the 1st ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
Imbalance optimization in scientific workflows
Proceedings of the 27th international ACM conference on International conference on supercomputing
| Hi-index | 0.00 |
The execution of scientific workflows often suffers from a variety of overheads in distributed environments. It is essential to identify the different overheads and to evaluate how optimization methods help reduce overheads and improve runtime performance. In this paper, we present an overhead analysis for a set of workflow runs on cloud and grid platforms. We present the overhead distributions and conclude that they satisfy an exponential or uniform distribution. We compare three methods to calculate the cumulative sum of the overheads based on how they overlap. In addition, we indicate how experimental parameters impact the overhead and thereby the overall workflow performance. We then show how popular optimization methods improve runtime performance by reducing some or all types of overheads.