The Art of Building a Good Benchmark

  • Authors:
  • Karl Huppler

  • Affiliations:
  • IBM Corporation, IBM MS XQK, Rochester 55901

  • Venue:
  • Performance Evaluation and Benchmarking
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

What makes a good benchmark? This is a question that has been asked often, answered often, altered often. In the past 25 years, the information processing industry has seen the creation of dozens of "industry standard" performance benchmarks --- some highly successful, some less so. This paper will explore the overall requirements of a good benchmark, using existing industry standards as examples along the way.