Benchmarking with Real Industrial Applications: The SPEC High-Performance Group

  • Authors:
  • Rudolf Eigenmann;Siamak Hassanzadeh

  • Affiliations:
  • -;-

  • Venue:
  • IEEE Computational Science & Engineering
  • Year:
  • 1996

Quantified Score

Hi-index 0.00

Visualization

Abstract

Simulations are only as good as the accuracy of the model. A truism, but one seemingly forgotten at times in the "benchmarking" of computer systems--the judging of their speed and performance running standardized programs. For really useful comparative benchmarking, not only must the conditions of the experiment be controlled, but the premises of the experiment must be valid. That is, the standard benchmark programs must be a reasonably good approximation of applications that real users might want to run on real machines under real working conditions. Too much simplification in the codes, and the results are of limited use in predicting performance.SPEC, the Standard Performance Evaluation Corporation, has been a leader in evaluating the performance of workstations since its formation in the late 1980s. Now a group within SPEC is turning its attention to real high-performance computing, and it's emphasizing the use of realistic scientific and industrial codes as benchmarks. A seismic code for oil exploration and a computational chemistry program are the first two components of the SPEChpc96 suite. SPEC hopes that interest and participation from users will help speed up the somewhat sluggish growth of real HPC performance.