Cost-Based Scheduling of Scientific Workflow Application on Utility Grids

Authors:
Jia Yu;Rajkumar Buyya;Chen Khong Tham
Affiliations:
University of Melbourne;University of Melbourne;National University of Singapore
Venue:
E-SCIENCE '05 Proceedings of the First International Conference on e-Science and Grid Computing
Year:
2005

Citing 0
Cited 52

Key research issues in grid workflow verification and validation

ACSW Frontiers '06 Proceedings of the 2006 Australasian workshops on Grid computing and e-research - Volume 54
Adaptive selection of necessary and sufficient checkpoints for dynamic verification of temporal constraints in grid workflow systems

ACM Transactions on Autonomous and Adaptive Systems (TAAS)
Scheduling scientific workflow applications with deadline and budget constraints using genetic algorithms

Scientific Programming - Scientific Workflows
INFORM: integrated flow orchestration and meta-scheduling for managed grid systems

Proceedings of the 2007 ACM/IFIP/USENIX international conference on Middleware companion
Economic aspects of a utility computing service

Proceedings of the first international conference on Networks for grid applications
Activity Completion Duration Based Checkpoint Selection for Dynamic Verification of Temporal Constraints in Grid Workflow Systems

International Journal of High Performance Computing Applications
The cost of doing science on the cloud: the Montage example

Proceedings of the 2008 ACM/IEEE conference on Supercomputing
Resource Allocation vs. Business Process Improvement: How They Impact on Each Other

BPM '08 Proceedings of the 6th International Conference on Business Process Management
Towards a general model of the multi-criteria workflow scheduling on the grid

Future Generation Computer Systems
DIPAS: A distributed performance analysis service for grid service-based workflows

Future Generation Computer Systems
Scientific workflow scheduling in computational grids Planning, reservation, and data/network-awareness

GRID '07 Proceedings of the 8th IEEE/ACM International Conference on Grid Computing
Deadline division-based heuristic for cost optimization in workflow scheduling

Information Sciences: an International Journal
An ant colony optimization approach to a grid workflow scheduling problem with various QoS requirements

IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
An ant colony optimization algorithm for the time-varying workflow scheduling problem in grids

CEC'09 Proceedings of the Eleventh conference on Congress on Evolutionary Computation
Resource Planning for Massive Number of Process Instances

OTM '09 Proceedings of the Confederated International Conferences, CoopIS, DOA, IS, and ODBASE 2009 on On the Move to Meaningful Internet Systems: Part I
Dynamic heuristics for time and cost reduction in grid workflows

CSCWD'06 Proceedings of the 10th international conference on Computer supported cooperative work in design III
A probability-based framework for dynamic resource scheduling in grid environment

GPC'08 Proceedings of the 3rd international conference on Advances in grid and pervasive computing
A collaborative scheduling approach for service-driven scientific workflow execution

Journal of Computer and System Sciences
Time and cost trade-off management for scheduling parallel applications on Utility Grids

Future Generation Computer Systems
Reshaping text data for efficient processing on Amazon EC2

Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing
An effective compaction strategy for bi-criteria DAG scheduling in grids

International Journal of Communication Networks and Distributed Systems
Scheduling parallel applications on utility grids: time and cost trade-off management

ACSC '09 Proceedings of the Thirty-Second Australasian Conference on Computer Science - Volume 91
A Min-Min average algorithm for scheduling transaction-intensive grid workflows

AusGrid '09 Proceedings of the Seventh Australasian Symposium on Grid Computing and e-Research - Volume 99
A marketplace for cloud resources

EMSOFT '10 Proceedings of the tenth ACM international conference on Embedded software
A Compromised-Time-Cost Scheduling Algorithm in SwinDeW-C for Instance-Intensive Cost-Constrained Workflows on a Cloud Computing Platform

International Journal of High Performance Computing Applications
A scheduling model for workflows on grids and clouds

Proceedings of the 8th International Workshop on Middleware for Grids, Clouds and e-Science
Evaluation of grid workflow scheduling techniques in dynamic grid simulation environments

Proceedings of the 8th International Conference on Frontiers of Information Technology
Scheduling large jobs by abstraction refinement

Proceedings of the sixth conference on Computer systems
Joint Elastic Cloud and Virtual Network Framework for Application Performance-cost Optimization

Journal of Grid Computing
BTS: Resource capacity estimate for time-targeted science workflows

Journal of Parallel and Distributed Computing
Auto-scaling to minimize cost and meet application deadlines in cloud workflows

Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis
Performance evaluation of scheduling algorithms for database services with soft and hard SLAs

Proceedings of the second international workshop on Data intensive computing in the clouds
Business-driven short-term management of a hybrid IT infrastructure

Journal of Parallel and Distributed Computing
Flexible service selection with user-specific QoS support in service-oriented architecture

Journal of Network and Computer Applications
A Multi-objective Approach for Workflow Scheduling in Heterogeneous Environments

CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
A bi-criteria scheduling process with CoS support on grids and clouds

Concurrency and Computation: Practice & Experience
Dynamic compatibility matching of services for distributed workflow execution

PPAM'11 Proceedings of the 9th international conference on Parallel Processing and Applied Mathematics - Volume Part II
Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds

Future Generation Computer Systems
Cost- and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds

SC '12 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
A Scheduling Algorithm for the Distributed Student Registration System in Transaction-Intensive Environment

International Journal of Distance Education Technologies
A Hybrid Resource Reservation Method for Workflows in Clouds

International Journal of Grid and High Performance Computing
CloudPack* exploiting workload flexibility through rational pricing

Proceedings of the 13th International Middleware Conference
A model, design, and implementation of an efficient multithreaded workflow execution engine with data streaming, caching, and storage constraints

The Journal of Supercomputing
A Hybrid Resource Reservation Method for Workflows in Clouds

International Journal of Grid and High Performance Computing
Trust services-oriented multi-objects workflow scheduling model for cloud computing

ICPCA/SWS'12 Proceedings of the 2012 international conference on Pervasive Computing and the Networked World
A continuous workflow scheduling framework

Proceedings of the 2nd ACM SIGMOD Workshop on Scalable Workflow Execution Engines and Technologies
Exploiting application dynamism and cloud elasticity for continuous dataflows

SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Reliable workflow scheduling with less resource redundancy

Parallel Computing
Cost adaptive workflow scheduling in cloud computing

Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication
Versatile time-cost algorithm VTCA for scheduling non-preemptive tasks of time critical workflows in cloud computing systems

International Journal of Communication Networks and Distributed Systems
Multi-objective list scheduling of workflow applications in distributed computing infrastructures

Journal of Parallel and Distributed Computing
BeesyBees: A mobile agent-based middleware for a reliable and secure execution of service-based workflow applications in BeesyCluster

Multiagent and Grid Systems - Agent Based Computing: From Model to Implementation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Over the last few years, Grid technologies have progressed towards a service-oriented paradigm that enables a new way of service provisioning based on utility computing models. Users consume these services based on their QoS (Quality of Service) requirements. In such "pay-per-use" Grids, workflow execution cost must be considered during scheduling based on users' QoS constraints. In this paper, we propose a cost-based workflow scheduling algorithm that minimizes execution cost while meeting the deadline for delivering results. It can also adapt to the delays of service executions by rescheduling unexecuted tasks. We also attempt to optimally solve the task scheduling problem in branches with several sequential tasks by modeling the branch as a Markov Decision Process and using the value iteration method.