Virtual Clusters on the Fly - Fast, Scalable, and Flexible Installation

  • Authors:
  • Hideo Nishimura;Naoya Maruyama;Satoshi Matsuoka

  • Affiliations:
  • Tokyo Institute of Technology;Tokyo Institute of Technology;National Institute of Informatics

  • Venue:
  • CCGRID '07 Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

One of the advantages in virtualized computing clusters compared to traditional shared HPC environments is their ability to accommodate user-specific system customization. However, past attempts to providing virtual clusters are not scalable with increasing number of VMs, nor do they allow fine-grained customization of VMs, assuming that preconfigured VM images are always available on the Grid. We propose a new virtual cluster installation technique that achieves efficiency and scalability, and yet simultaneously fine-grained customizability. It allows the user to create VMs on the fly for fine-grained customization of VMs, and pipelined data transfer for scalable installation with increasing number of VMs. To achieve efficiency in the presence of such full customization, it automatically caches frequently-constructed virtual disk images to save software installation time in common cases. Our experimental studies using a prototype implementation show that installation of a 190-node virtual cluster can be done in 40 seconds. From this result along with a scalability study, we estimate that installation of a 1000-node virtual cluster could be done in less than two minutes.