SPECrate2006: Alternatives Considered, Lessons Learned

Authors:
John L. Henning
Affiliations:
Sun Microsystems,
Venue:
Proceedings of the 2009 SPEC Benchmark Workshop on Computer Performance Evaluation and Benchmarking
Year:
2009

Citing 4
Cited 0

The visual display of quantitative information

The visual display of quantitative information
SPEC CPU suite growth: an historical perspective

ACM SIGARCH Computer Architecture News
CPU2006 working set size

ACM SIGARCH Computer Architecture News
Subroutine profiling results for the CPU2006 benchmarks

ACM SIGARCH Computer Architecture News

Quantified Score

Hi-index	0.00

Visualization

Abstract

Since 1992, SPEC has used multiple identical benchmarks to measure multi-processor performance. This "Homogeneous Capacity Method" (aka "SPECrate") has been criticized on the grounds that real workloads are not homogeneous. Nevertheless, SPECrate provides a useful window into how systems perform when stressed by multiple requests for similar resources. This paper reviews SPECrate's history, and several performance lessons learned using it: (1) a 4:1 performance gain for startup of a benchmark when I/O was reconfigured; (2) a benchmark that improved up to 2:1 when a TLB data structure was re-sized; and (3) a benchmark that improved by 52% after a change to NUMA page allocation. The SPEC CPU workloads usefully exposed several opportunities for performance improvement.