Predicting the execution time of grid workflow applications through local learning

Authors:
Farrukh Nadeem;Thomas Fahringer
Affiliations:
National University of Computer and Emerging Sciences, Lahore, Pakistan and University of Innsbruck, Austria;University of Innsbruck, Austria
Venue:
Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis
Year:
2009

Citing 19
Cited 8

A performance model of block structured parallel programs

Proceedings of the international workshop on Parallel algorithms & architectures
Locally Weighted Learning

Artificial Intelligence Review - Special issue on lazy learning
Multi-Objective Optimization Using Evolutionary Algorithms

Multi-Objective Optimization Using Evolutionary Algorithms
Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing

IEEE Transactions on Parallel and Distributed Systems
Job Characteristics of a Production Parallel Scientivic Workload on the NASA Ames iPSC/860

IPPS '95 Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing
Evaluation of parallel execution of program tree structures

SIGMETRICS '84 Proceedings of the 1984 ACM SIGMETRICS conference on Measurement and modeling of computer systems
Using Kernel Couplings to Predict Parallel Application Performance

HPDC '02 Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing
Predicting application run times with historical information

Journal of Parallel and Distributed Computing
Distributed computing with Triana on the Grid: Research Articles

Concurrency and Computation: Practice & Experience
Taverna: a tool for the composition and enactment of bioinformatics workflows

Bioinformatics
Improving a Local Learning Technique for QueueWait Time Predictions

CCGRID '06 Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid
Contention-sensitive static performance prediction for parallel distributed applications

Performance Evaluation
Piloting an Empirical Study on Measures forWorkflow Similarity

SCC '06 Proceedings of the IEEE International Conference on Services Computing
Soft Benchmarks-Based Application Performance Prediction Using a Minimum Training Set

E-SCIENCE '06 Proceedings of the Second IEEE International Conference on e-Science and Grid Computing
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
Pegasus: A framework for mapping complex scientific workflows onto distributed systems

Scientific Programming
Advanced data flow support for scientific grid workflow applications

Proceedings of the 2007 ACM/IEEE conference on Supercomputing
A Probabilistic Model to Analyse Workflow Performance on Production Grids

CCGRID '08 Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid
Improved heterogeneous distance functions

Journal of Artificial Intelligence Research

PV-EASY: a strict fairness guaranteed and prediction enabled scheduler in parallel job scheduling

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
A novel statistical time-series pattern based interval forecasting strategy for activity durations in workflow systems

Journal of Systems and Software
TRACON: interference-aware scheduling for data-intensive applications in virtualized environments

Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
Failure prediction and localization in large scientific workflows

Proceedings of the 6th workshop on Workflows in support of large-scale science
Flexible service selection with user-specific QoS support in service-oriented architecture

Journal of Network and Computer Applications
A Multi-objective Approach for Workflow Scheduling in Heterogeneous Environments

CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
Execution time prediction for grid infrastructures based on runtime provenance data

WORKS '13 Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science
A framework for dynamically generating predictive models of workflow execution

WORKS '13 Proceedings of the 8th Workshop on Workflows in Support of Large-Scale Science

Quantified Score

Hi-index	0.00

Visualization

Abstract

Workflow execution time prediction is widely seen as a key service to understand the performance behavior and support the optimization of Grid workflow applications. In this paper, we present a novel approach for estimating the execution time of workflows based on Local Learning. The workflows are characterized in terms of different attributes describing structural and runtime information about workflow activities, control and data flow dependencies, number of Grid sites, problem size, etc. Our local learning framework is complemented by a dynamic weighing scheme that assigns weights to workflow attributes reflecting their impact on the workflow execution time. Predictions are given through intervals bounded by the minimum and maximum predicted values, which are associated with a confidence value indicating the degree of confidence about the prediction accuracy. Evaluation results for three real world workflows on a real Grid are presented to demonstrate the prediction accuracy and overheads of the proposed method.