A framework for reliable and efficient data placement in distributed computing systems
Journal of Parallel and Distributed Computing - Special issue: Design and performance of networks for super-, cluster-, and grid-computing: Part I
MapReduce: simplified data processing on large clusters
Communications of the ACM - 50th anniversary issue: 1958 - 2008
Bioinformatics
On the energy (in)efficiency of Hadoop clusters
ACM SIGOPS Operating Systems Review
Improving MapReduce performance in heterogeneous environments
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Energy management for MapReduce clusters
Proceedings of the VLDB Endowment
GreenHDFS: towards an energy-conserving, storage-efficient, hybrid Hadoop compute cluster
HotPower'10 Proceedings of the 2010 international conference on Power aware computing and systems
MARIANE: MApReduce Implementation Adapted for HPC Environments
GRID '11 Proceedings of the 2011 IEEE/ACM 12th International Conference on Grid Computing
TAPA: Temperature aware power allocation in data center with Map-Reduce
IGCC '11 Proceedings of the 2011 International Green Computing Conference and Workshops
Improving MapReduce energy efficiency for computation intensive workloads
IGCC '11 Proceedings of the 2011 International Green Computing Conference and Workshops
MARLA: MapReduce for Heterogeneous Clusters
CCGRID '12 Proceedings of the 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012)
| Hi-index | 0.00 |
MapReduce has become a popular framework for Big Data applications. While MapReduce has received much praise for its scalability and efficiency, it has not been thoroughly evaluated for power consumption. Our goal with this paper is to explore the possibility of scheduling in a power-efficient manner without the need for expensive power monitors on every node. We begin by considering that no cluster is truly homogeneous with respect to energy consumption. From there we develop a MapReduce framework that can evaluate the current status of each node and dynamically react to estimated power usage. In so doing, we shift work toward more energy efficient nodes which are currently consuming less power. Our work shows that given an ideal framework configuration, certain nodes may consume only 62.3 % of the dynamic power they consumed when the same framework was configured as it would be in a traditional MapReduce implementation.