Trusting floating point benchmarks - are your benchmarks really data independent?

  • Authors:

  • John Markus Bjørndalen;Otto J. Anshus

  • Affiliations:

  • Department of Computer Science, University of Tromsø, Tromsø, Norway;Department of Computer Science, University of Tromsø, Tromsø, Norway

  • Venue:
  • PARA'06 Proceedings of the 8th international conference on Applied parallel computing: state of the art in scientific computing
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Benchmarks are important tools for studying increasingly complex hardware architectures and software systems. Two seemingly common assumptions are that the execution time of floating point operations do not change much with different input values, and that the execution time of a benchmark does not vary much if the input and computed values do not influence any branches. These assumption do not always hold. There is significant overhead in handling denormalized floating point values (a representation automatically used by the CPU to represent values close to zero) on-chip on modern Intel hardware, even if the program can continue uninterrupted. We have observed that even a small fraction of denormal numbers in a textbook benchmark significantly increases the execution time of the benchmark, leading to the wrong conclusions about the relative efficiency of different hardware architectures and about scalability problems of a cluster benchmark.