Evolution and evaluation of SPEC benchmarks

Authors:
Jozo J. Dujmovic;Ivo Dujmovic
Affiliations:
Department of Computer Science, San Francisco State University;Performance and Tuning Group, Oracle Corporation
Venue:
ACM SIGMETRICS Performance Evaluation Review
Year:
1998

Citing 0
Cited 15

Array resizing for scientific code debugging, maintenance and reuse

PASTE '01 Proceedings of the 2001 ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering
Adapting the SPEC 2000 benchmark suite for simulation-based computer architecture research

Workload characterization of emerging computer applications
Efficient and effective array bound checking

ACM Transactions on Programming Languages and Systems (TOPLAS)
Measuring Benchmark Similarity Using Inherent Program Characteristics

IEEE Transactions on Computers
The exigency of benchmark and compiler drift: designing tomorrow's processors with yesterday's tools

Proceedings of the 20th annual international conference on Supercomputing
Analysis of redundancy and application balance in the SPEC CPU2006 benchmark suite

Proceedings of the 34th annual international symposium on Computer architecture
Evaluating the performance of single and multiple core processors with PCMARK®05 and benchmark analysis

ACM SIGMETRICS Performance Evaluation Review
Design and implementation of a queue compiler

Microprocessors & Microsystems
Efficient compilation for queue size constrained queue processors

Parallel Computing
Finding representative workloads for computer system design

Finding representative workloads for computer system design
Automatic generation of benchmark and test workloads

Proceedings of the first joint WOSP/SIPEW international conference on Performance engineering
Compiling for Reduced Bit-Width Queue Processors

Journal of Signal Processing Systems
Automatic detection of uninitialized variables

CC'03 Proceedings of the 12th international conference on Compiler construction
Pruning hardware evaluation space via correlation-driven application similarity analysis

Proceedings of the 8th ACM International Conference on Computing Frontiers
Natural instruction level parallelism-aware compiler for high-performance QueueCore processor architecture

The Journal of Supercomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

We present a method for quantitative evaluation of SPEC benchmarks. The method is used for the analysis of three generations of SPEC component-level benchmarks: SPEC89, SPEC92, and SPEC95. Our approach is suitable for studying (1) the redundancy between individual benchmark programs, (2) the size, completeness, density and granularity of benchmark suites, (3) the distribution of benchmark programs in a program space, and (4) benchmark suite design and evolution strategies. The presented method can be used for designing a universal benchmark suite as the next generation of SPEC benchmarks.