To compress or not to compress - compute vs. IO tradeoffs for mapreduce energy efficiency

Authors:
Yanpei Chen;Archana Ganapathi;Randy H. Katz
Affiliations:
University of California, Berkeley, Berkeley, USA;University of California, Berkeley, Berkeley, USA;University of California, Berkeley, Berkeley, USA
Venue:
Proceedings of the first ACM SIGCOMM workshop on Green networking
Year:
2010

Citing 2
Cited 8

JouleSort: a balanced energy-efficiency benchmark

Proceedings of the 2007 ACM SIGMOD international conference on Management of data
The Case for Energy-Proportional Computing

Computer

Scarlett: coping with skewed content popularity in mapreduce clusters

Proceedings of the sixth conference on Computer systems
Parallel data processing with MapReduce: a survey

ACM SIGMOD Record
Compression-aware I/O performance analysis for big data clustering

Proceedings of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications
Application-driven energy-efficient architecture explorations for big data

Proceedings of the 1st Workshop on Architectures and Systems for Big Data
Modeling I/O interference for data intensive distributed applications

Proceedings of the 28th Annual ACM Symposium on Applied Computing
Energy efficiency for MapReduce workloads: an in-depth study

ADC '12 Proceedings of the Twenty-Third Australasian Database Conference - Volume 124
Exploiting MapReduce and data compression for data-intensive applications

Proceedings of the Conference on Extreme Science and Engineering Discovery Environment: Gateway to Discovery
Scalable hybrid stream and hadoop network analysis system

Proceedings of the 5th ACM/SPEC international conference on Performance engineering

Quantified Score

Hi-index	0.00

Visualization

Abstract

Compression enables us to shift resource bottlenecks between IO and CPU. In modern datacenters, where energy efficiency is a growing concern, the benefits of using compression have not been completely exploited. As MapReduce represents a common computation framework for Internet datacenters, we develop a decision algorithm that helps MapReduce users identify when and where to use compression. For some jobs, using compression gives energy savings of up to 60%. We believe our findings will provide signficant impact on improving datacenter energy efficiency.