Stochastic Tail-Phase Optimization for Bag-of-Tasks Execution in Clouds
UCC '12 Proceedings of the 2012 IEEE/ACM Fifth International Conference on Utility and Cloud Computing
Fast (re-)configuration of mixed on-demand and spot instance pools for high-throughput computing
Proceedings of the first ACM workshop on Optimization techniques for resources management in clouds
Exploring portfolio scheduling for long-term execution of scientific workloads in IaaS clouds
SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis
Scheduling jobs in the cloud using on-demand and reserved instances
Euro-Par'13 Proceedings of the 19th international conference on Parallel Processing
| Hi-index | 0.00 |
Many scientists perform extensive computations by executing large bags of similar tasks (BoTs) in mixtures of computational environments, such as grids and clouds. Although the reliability and cost may vary considerably across these environments, no tool exists to assist scientists in the selection of environments that can both fulfill deadlines and fit budgets. To address this situation, we introduce the Expert BoT scheduling framework. Our framework systematically selects from a large search space the Pareto-efficient scheduling strategies, that is, the strategies that deliver the best results for both make span and cost. Expert chooses from them the best strategy according to a general, user-specified utility function. Through simulations and experiments in real production environments, we demonstrate that Expert can substantially reduce both make span and cost in comparison to common scheduling strategies. For bioinformatics BoTs executed in a real mixed grid + cloud environment, we show how the scheduling strategy selected by Expert reduces both make span and cost by 30%-70%, in comparison to commonly-used scheduling strategies.