Energy efficiency for MapReduce workloads: an in-depth study

Authors:
Boliang Feng;Jiaheng Lu;Yongluan Zhou;Nan Yang
Affiliations:
Renmin University of China, Beijing, PR China;Renmin University of China, Beijing, PR China;University of Southern Denmark, Campusvej, Odense M;Renmin University of China, Beijing, PR China
Venue:
ADC '12 Proceedings of the Twenty-Third Australasian Database Conference - Volume 124
Year:
2012

Citing 15
Cited 1

Managing energy and server resources in hosting centers

SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
On evaluating request-distribution schemes for saving energy in server clusters

ISPASS '03 Proceedings of the 2003 IEEE International Symposium on Performance Analysis of Systems and Software
JouleSort: a balanced energy-efficiency benchmark

Proceedings of the 2007 ACM SIGMOD international conference on Management of data
MapReduce: simplified data processing on large clusters

OSDI'04 Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6
The Case for Energy-Proportional Computing

Computer
Database servers tailored to improve energy efficiency

SETMDM '08 Proceedings of the 2008 EDBT workshop on Software engineering for tailor-made data management
Energy cost, the key challenge of today's data centers: a power consumption analysis of TPC-C results

Proceedings of the VLDB Endowment
PowerNap: eliminating server idle power

Proceedings of the 14th international conference on Architectural support for programming languages and operating systems
On the energy (in)efficiency of Hadoop clusters

ACM SIGOPS Operating Systems Review
Energy-efficient server clusters

PACS'02 Proceedings of the 2nd international conference on Power-aware computer systems
Analyzing the energy efficiency of a database server

Proceedings of the 2010 ACM SIGMOD International Conference on Management of data
To compress or not to compress - compute vs. IO tradeoffs for mapreduce energy efficiency

Proceedings of the first ACM SIGCOMM workshop on Green networking
FAWNdamentally power-efficient clusters

HotOS'09 Proceedings of the 12th conference on Hot topics in operating systems
Wimpy node clusters: what about non-wimpy workloads?

Proceedings of the Sixth International Workshop on Data Management on New Hardware
Energy management for MapReduce clusters

Proceedings of the VLDB Endowment

Analyzing job completion reliability and job energy consumption for a general MapReduce infrastructure

Journal of High Speed Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Energy efficiency has emerged as a crucial optimization goal in data centers. MapReduce has become a popular and even fashionable distributed processing model for parallel computing in data centers. Hadoop is an open-source implementation of MapReduce, which is widely used for short jobs requiring low response time. In this paper, we conduct an in-depth study of the energy efficiency for MapReduce workloads. We identify four factors that affect the energy efficiency of MapReduce. In particular, we make experiments over four typical MapReduce workloads that represent different kinds of application scenarios and measure the energy consumption with varied cluster parameters. Our key finding is that with well-tuned system parameters and adaptive resource configurations, MapReduce cluster can achieve both performance improvement and good energy saving simultaneously in some instances, which is surprisingly contrast to previous works on cluster-level energy conservation.