Workload diversity and dynamics in big data analytics: implications to system designers

Authors:
Jichuan Chang;Kevin T. Lim;John Byrne;Laura Ramirez;Parthasarathy Ranganathan
Affiliations:
Hewlett Packard Labs, Palo Alto;Hewlett Packard Labs, Palo Alto;Hewlett Packard Labs, Palo Alto;Hewlett Packard Labs, Palo Alto;Hewlett Packard Labs, Palo Alto
Venue:
Proceedings of the 2nd Workshop on Architectures and Systems for Big Data
Year:
2012

Citing 9
Cited 0

Understanding and Designing New Server Architectures for Emerging Warehouse-Computing Environments

ISCA '08 Proceedings of the 35th Annual International Symposium on Computer Architecture
PowerNap: eliminating server idle power

Proceedings of the 14th international conference on Architectural support for programming languages and operating systems
Capture, conversion, and analysis of an intense NFS workload

FAST '09 Proccedings of the 7th conference on File and storage technologies
FAWN: a fast array of wimpy nodes

Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles
Server Engineering Insights for Large-Scale Online Services

IEEE Micro
The mixed workload CH-benCHmark

Proceedings of the Fourth International Workshop on Testing Database Systems
HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots

ICDE '11 Proceedings of the 2011 IEEE 27th International Conference on Data Engineering
LazyBase: trading freshness for performance in a scalable database

Proceedings of the 7th ACM european conference on Computer Systems
Many-core key-value store

IGCC '11 Proceedings of the 2011 International Green Computing Conference and Workshops

Quantified Score

Hi-index	0.00

Visualization

Abstract

The emergence of big data analytics and the need for cost/energy efficient IT infrastructure motivate a new focus on data-centric designs. In this paper, we aim to better understand the design implications of data analytics systems by quantifying workload requirements and runtime dynamics. We examine four workloads representing big data analytics trends for fast decisions, total integration, deep analysis and fresh insights: an archive store, a columnar database enhanced with table compression, an analytics engine with distributed R, and a transaction/analytics hybrid system. These appliations demonstrate diverse resource requirements both within and across workloads as well as load imbalance due to data skew. Our observations suggest several directions to design balanced data analytics systems, including tight integration of heterogeneous, active data stores, support for efficient communication and data-centric load balancing.