Evaluating high-performance computers: Research Articles

  • Authors:
  • Jeffrey S. Vetter;Bronis R. de Supinski;Lynn Kissel;John May;Sheila Vaidya

  • Affiliations:
  • Oak Ridge National Laboratory, Oak Ridge, TN 37922, U.S.A.;Lawrence Livermore National Laboratory, Livermore, CA 94551, U.S.A.;Lawrence Livermore National Laboratory, Livermore, CA 94551, U.S.A.;Lawrence Livermore National Laboratory, Livermore, CA 94551, U.S.A.;Lawrence Livermore National Laboratory, Livermore, CA 94551, U.S.A.

  • Venue:
  • Concurrency and Computation: Practice & Experience - The High Performance Architectural Challenge: Mass Market versus Proprietary Components?
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

Comparisons of high-performance computers based on their peak floating point performance are common but seldom useful when comparing performance on real workloads. Factors that influence sustained performance extend beyond a system's floating-point units, and real applications exercise machines in complex and diverse ways. Even when it is possible to compare systems based on their performance, other considerations affect which machine is best for a given organization. These include the cost, the facilities requirements (power, floorspace, etc.), the programming model, the existing code base, and so on. This paper describes some of the important measures for evaluating high-performance computers. We present data for many of these metrics based on our experience at Lawrence Livermore National Laboratory (LLNL), and we compare them with published information on the Earth Simulator. We argue that evaluating systems involves far more than comparing benchmarks and acquisition costs. We show that evaluating systems often involves complex choices among a variety of factors that influence the value of a supercomputer to an organization, and that the high-end computing community should view cost/performance comparisons of different architectures with skepticism. Published in 2005 by John Wiley & Sons, Ltd.This article is a U.S. Government work and is in the public domain in the U.S.A.