Performance Comparison of Coscheduling Algorithms for Non-Dedicated Clusters Through a Generic Framework

  • Authors:

  • Gyu Sang Choi;Saurabh Agarwal;Jin-Ha Kim;Chita R. Das;Andy B. Yoo

  • Affiliations:

  • Samsung Advanced Institute of Technology, Samsung Electronics, MT. 14-1, Nong-Seo-Dong, Giheung-Gu, Yongin-Si, Gyeonggi-Do, Korea 446-712;Ross School of Business, University of Michigan, 701 Tappan Street, Ann Arbor, MI 48109;Samsung Networks, 8F, Asem Tower, World Trade Center, 159-1, Samsung-Dong, Kangnam-Ku, Seoul, Korea 135-798;The Department of Computer Science and Engineering, The Pennsylvania State University, University Park, PA 16802;Lawrence Livermore National Laboratory, Livermore, CA 94551

  • Venue:
  • International Journal of High Performance Computing Applications
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, we address several key issues in designing coscheduling algorithms for clusters. First, we propose a generic framework for deploying coscheduling techniques by providing a reusable and dynamically loadable kernel module. Second, we implement several communication-driven coscheduling algorithms [dynamic coscheduling (DCS), spin block (SB) and periodic boost (PB)] on a 16- node Linux cluster using the above framework. Third, with exhaustive experimentation using mixed workloads, we observe that unlike PB, which provided the best performance on a Solaris platform, the SB scheme outperforms all other techniques on a Linux platform. Finally, we investigate the impact of several job placement strategies, multiprogramming level (MPL), communication intensity and CPU and I/O intensive jobs on the performance of these coscheduling schemes. The experimental results show that the blocking-based coscheduling scheme (SB) has better tolerance to system workload variation compared with the spin-based schemes (DCS and PB), and overall, the blocking-based coscheduling scheme seems a better choice for non-dedicated Linux clusters.