Evaluating the impact of the simulation environment on experimentation results

Authors:
Lambert Schaelicke
Affiliations:
Department of Computer Science and Engineering, University of Notre Dame, 384 Fitzpatrick Hall, Notre Dame, IN 46556, USA
Venue:
Performance Evaluation
Year:
2005

Citing 11
Cited 0

Memory system performance of programs with intensive heap allocation

ACM Transactions on Computer Systems (TOCS)
Using the SimOS machine simulator to study complex computer systems

ACM Transactions on Modeling and Computer Simulation (TOMACS)
FLASH vs. (Simulated) FLASH: closing the simulation loop

ASPLOS IX Proceedings of the ninth international conference on Architectural support for programming languages and operating systems
Measuring Experimental Error in Microprocessor Simulation

ISCA '01 Proceedings of the 28th annual international symposium on Computer architecture
Calibration of Microprocessor Performance Models

Computer
SPEC CPU2000: Measuring CPU Performance in the New Millennium

Computer
RSIM: Simulating Shared-Memory Multiprocessors with ILP Processors

Computer
Simics: A Full System Simulation Platform

Computer
SimpleScalar: An Infrastructure for Computer System Modeling

Computer
Can Trace-Driven Simulators Accurately Predict Superscalar Performance?

ICCD '96 Proceedings of the 1996 International Conference on Computer Design, VLSI in Computers and Processors
Improving the Accuracy vs. Speed Tradeoff for Simulating Shared-Memory Multiprocessors with ILP Processors

HPCA '99 Proceedings of the 5th International Symposium on High Performance Computer Architecture

Quantified Score

Hi-index	0.00

Visualization

Abstract

Simulation has become the tool of choice for advanced computer architecture research. Simulation model abstraction as well as compiler specifics can have significant impact on the observed performance trends and may lead to incorrect conclusions. This study presents a methodology that compares the predicted performance impact of several architectural techniques measured on a variety of simulator and compiler configurations. A significant difference in speedup between experiments indicates that a technique is sensitive to this particular variation. The results demonstrate a strong but non-uniform relationship between modeling abstraction and measurement error, as well as a significant sensitivity to the compiler.