Metrics for measuring the performance of the mixed workload CH-benCHmark

  • Authors:

  • Florian Funke;Alfons Kemper;Stefan Krompass;Harumi Kuno;Raghunath Nambiar;Thomas Neumann;Anisoara Nica;Meikel Poess;Michael Seibold

  • Affiliations:

  • Technische Universität München, Garching bei München, Germany;Technische Universität München, Garching bei München, Germany;Technische Universität München, Garching bei München, Germany;HP Labs, Palo Alto, CA;Cisco Systems, Inc., San Jose, CA;Technische Universität München, Garching bei München, Germany;Sybase, An SAP Company, Waterloo, ON, Canada;Oracle Corporation, Redwood Shores, CA;Technische Universität München, Garching bei München, Germany

  • Venue:
  • TPCTC'11 Proceedings of the Third TPC Technology conference on Topics in Performance Evaluation, Measurement and Characterization
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Advances in hardware architecture have begun to enable database vendors to process analytical queries directly on operational database systems without impeding the performance of mission-critical transaction processing too much. In order to evaluate such systems, we recently devised the mixed workload CH-benCHmark, which combines transactional load based on TPC-C order processing with decision support load based on TPC-H-like query suite run in parallel on the same tables in a single database system. Just as the data volume of actual enterprises tends to increase over time, an inherent characteristic of this mixed workload benchmark is that data volume increases during benchmark runs, which in turn may increase response times of analytic queries. For purely transactional loads, response times typically do not depend that much on data volume, as the queries used within business transactions are less complex and often indexes are used to answer these queries with point-wise accesses only. But for mixed workloads, the insert throughput metric of the transactional component interferes with the response-time metric of the analytic component. In order to address the problem, in this paper we analyze the characteristics of CH-benCHmark queries and propose normalized metrics which account for data volume growth.