Evaluating MapReduce on Virtual Machines: The Hadoop Case

  • Authors:

  • Shadi Ibrahim;Hai Jin;Lu Lu;Li Qi;Song Wu;Xuanhua Shi

  • Affiliations:

  • Cluster and Grid Computing Lab Services Computing Technology and System Lab, Huazhong University of Science & Technology, Wuhan, China 430074;Cluster and Grid Computing Lab Services Computing Technology and System Lab, Huazhong University of Science & Technology, Wuhan, China 430074;Cluster and Grid Computing Lab Services Computing Technology and System Lab, Huazhong University of Science & Technology, Wuhan, China 430074;Operation Center, China Development Bank, Beijing, China;Cluster and Grid Computing Lab Services Computing Technology and System Lab, Huazhong University of Science & Technology, Wuhan, China 430074;Cluster and Grid Computing Lab Services Computing Technology and System Lab, Huazhong University of Science & Technology, Wuhan, China 430074

  • Venue:
  • CloudCom '09 Proceedings of the 1st International Conference on Cloud Computing
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

MapReduceis emerging as an important programming model for large scale parallel application. Meanwhile, Hadoop is an open source implementation of MapReduce enjoying wide popularity for developing data intensive applications in the cloud. As, in the cloud, the computing unit is virtual machine (VM) based; it is feasible to demonstrate the applicability of MapReduce on virtualized data center. Although the potential for poor performance and heavy load no doubt exists, virtual machines can instead be used to fully utilize the system resources, ease the management of such systems, improve the reliability, and save the power. In this paper, a series of experiments are conducted to measure and analyze the performance of Hadoop on VMs. Our experiments are used as a basis for outlining several issues that will need to be considered when implementing MapReduce to fit completely in the cloud.