Improving the performance of I/O-intensive applications on clusters of workstations

  • Authors:
  • Xiao Qin;Hong Jiang;Yifeng Zhu;David R. Swanson

  • Affiliations:
  • Department of Computer Science, New Mexico Institute of Mining and Technology, Socorro, USA 87801-4796;Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA 68588-0115;Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA 68588-0115;Department of Computer Science and Engineering, University of Nebraska-Lincoln, Lincoln, USA 68588-0115

  • Venue:
  • Cluster Computing
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Load balancing in a workstation-based cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies can cause system performance to decrease substantially. In this paper, two I/O-aware load-balancing schemes, referred to as IOCM and WAL-PM, are presented to improve the overall performance of a cluster system with a general and practical workload including I/O activities. The proposed schemes dynamically detect I/O load imbalance of nodes in a cluster, and determine whether to migrate some I/O load from overloaded nodes to other less- or under-loaded nodes. The current running jobs are eligible to be migrated in WAL-PM only if overall performance improves. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in the system, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Extensive trace-driven simulations for both synthetic and real I/O-intensive applications show that: (1) Compared with existing schemes that only consider CPU and memory, the proposed schemes improve the performance with respect to mean slowdown by up to a factor of 20; (2) When compared to the existing approaches that only consider I/O with non-preemptive job migrations, the proposed schemes achieve improvements in mean slowdown by up to a factor of 10; (3) Under CPU-memory intensive workloads, our scheme improves the performance over the existing approaches that only consider I/O by up to 47.5%.