Bigtable: a distributed storage system for structured data
OSDI '06 Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation - Volume 7
MapReduce: simplified data processing on large clusters
Communications of the ACM - 50th anniversary issue: 1958 - 2008
pMapper: power and migration cost aware application placement in virtualized systems
Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware
PowerNap: eliminating server idle power
Proceedings of the 14th international conference on Architectural support for programming languages and operating systems
FAWN: a fast array of wimpy nodes
Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles
On the energy (in)efficiency of Hadoop clusters
ACM SIGOPS Operating Systems Review
An energy case for hybrid datacenters
ACM SIGOPS Operating Systems Review
Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling
Proceedings of the 5th European conference on Computer systems
Energy aware consolidation for cloud computing
HotPower'08 Proceedings of the 2008 conference on Power aware computing and systems
Improving MapReduce performance in heterogeneous environments
OSDI'08 Proceedings of the 8th USENIX conference on Operating systems design and implementation
Wimpy node clusters: what about non-wimpy workloads?
Proceedings of the Sixth International Workshop on Data Management on New Hardware
Energy management for MapReduce clusters
Proceedings of the VLDB Endowment
GreenHDFS: towards an energy-conserving, storage-efficient, hybrid Hadoop compute cluster
HotPower'10 Proceedings of the 2010 international conference on Power aware computing and systems
| Hi-index | 0.00 |
Energy efficiency has become the center of attention in emerging data center infrastructures as increasing energy costs continue to outgrow all other operating expenditures. In this work we investigate energy aware scheduling heuristics to increase the energy efficiency of MapReduce workloads on heterogeneous Hadoop clusters comprising both low power (wimpy) and high performance (brawny) nodes. We first make a case for heterogeneity by showing that low power Intel Atom processors and high performance Intel Sandy Bridge processors are more energy efficient for I/O bound workloads and CPU bound workloads, respectively. Then we present several energy efficient scheduling heuristics that exploit this heterogeneity and real-time power measurements enabled by modern processor architectures. Through experiments on a 23-node heterogeneous Hadoop cluster we demonstrate up to 27% better energy efficiency with our heuristics compared with the default Hadoop scheduler.