Toward real-time performance benchmarks for Ada
Communications of the ACM
Computer benchmarking: paths and pitfalls
IEEE Spectrum
Journal of the ACM (JACM)
Performance of various computers using standard linear equations software in a FORTRAN environment
ACM SIGARCH Computer Architecture News
Dhrystone benchmark: rationale for version 2 and measurement rules
ACM SIGPLAN Notices
An overview of the PTRAN analysis system for multiprocessing
Proceedings of the 1st International Conference on Supercomputing
Determining average program execution times and their variance
PLDI '89 Proceedings of the ACM SIGPLAN 1989 Conference on Programming language design and implementation
The parascope editor: an interactive parallel programming tool
Proceedings of the 1989 ACM/IEEE conference on Supercomputing
Machine Characterization Based on an Abstract High-Level Language Machine
IEEE Transactions on Computers
A static performance estimator to guide data partitioning decisions
PPOPP '91 Proceedings of the third ACM SIGPLAN symposium on Principles and practice of parallel programming
CPU performance evaluation and execution time prediction using narrow spectrum benchmarking
CPU performance evaluation and execution time prediction using narrow spectrum benchmarking
Compiler transformations for high-performance computing
ACM Computing Surveys (CSUR)
Micro-Analysis of Computer System Performance
Micro-Analysis of Computer System Performance
Measuring Cache and TLB Performance and Their Effect on Benchmark Runtimes
IEEE Transactions on Computers
Performance Characterization of Optimizing Compilers
IEEE Transactions on Software Engineering
Analysis of computational systems: Discrete Markov analysis of computer programs
ACM '65 Proceedings of the 1965 20th national conference
An instruction timing model of CPU performance
ISCA '77 Proceedings of the 4th annual symposium on Computer architecture
Cache Performance of the SPEC Benchmark Suite
Cache Performance of the SPEC Benchmark Suite
Analysis of Benchmark Characteristics and Benchmark Performance
Analysis of Benchmark Characteristics and Benchmark Performance
Microbenchmarking and Performance Prediction for Parallel
Microbenchmarking and Performance Prediction for Parallel
Machine Characterization and Benchmark Performance Prediction
Machine Characterization and Benchmark Performance Prediction
Predicting parallel applications performance on non-dedicated cluster platforms
ICS '98 Proceedings of the 12th international conference on Supercomputing
HBench:Java: an application-specific benchmarking framework for Java virtual machines
Proceedings of the ACM 2000 conference on Java Grande
Expressing meaningful processing requirements among heterogeneous nodes in an active network
Proceedings of the 2nd international workshop on Software and performance
Automatic time-bound analysis for a higher-order language
PEPM '02 Proceedings of the 2002 ACM SIGPLAN workshop on Partial evaluation and semantics-based program manipulation
Performance prediction for random write reductions: a case study in modeling shared memory programs
SIGMETRICS '02 Proceedings of the 2002 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Automatic Accurate Cost-Bound Analysis for High-Level Languages
IEEE Transactions on Computers
Workload Design: Selecting Representative Program-Input Pairs
Proceedings of the 2002 International Conference on Parallel Architectures and Compilation Techniques
Impact of PCI-Bus Load on Applications in a PC Architecture
RTSS '03 Proceedings of the 24th IEEE International Real-Time Systems Symposium
Application of the operational profile in software performance analysis
WOSP '04 Proceedings of the 4th international workshop on Software and performance
EXPERT: expedited simulation exploiting program behavior repetition
Proceedings of the 18th annual international conference on Supercomputing
Using Information from Prior Runs to Improve Automated Tuning Systems
Proceedings of the 2004 ACM/IEEE conference on Supercomputing
EMPS: An Environment for Memory Performance Studies
IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Workshop 10 - Volume 11
Application of redundant computation in software performance analysis
Proceedings of the 5th international workshop on Software and performance
How Well Can Simple Metrics Represent the Performance of HPC Applications?
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
A performance prediction framework for scientific applications
Future Generation Computer Systems
Measuring Benchmark Similarity Using Inherent Program Characteristics
IEEE Transactions on Computers
Performance prediction based on inherent program similarity
Proceedings of the 15th international conference on Parallel architectures and compilation techniques
IEEE Transactions on Computers
A genetic algorithms approach to modeling the performance of memory-bound computations
Proceedings of the 2007 ACM/IEEE conference on Supercomputing
Distilling the essence of proprietary workloads into miniature benchmarks
ACM Transactions on Architecture and Code Optimization (TACO)
A dollar from 15 cents: cross-platform management for internet services
ATC'08 USENIX 2008 Annual Technical Conference on Annual Technical Conference
P-Ray: A Software Suite for Multi-core Architecture Characterization
Languages and Compilers for Parallel Computing
Performance modeling for DFT algorithms in FFTW
Proceedings of the 23rd international conference on Supercomputing
Information Technology and Management
Finding representative workloads for computer system design
Finding representative workloads for computer system design
A performance prediction framework for scientific applications
Future Generation Computer Systems
Performance modeling for dynamic algorithm selection
ICCS'03 Proceedings of the 2003 international conference on Computational science
From source code to runtime behaviour: Software metrics help to select the computer architecture
Knowledge-Based Systems
Seekable sockets: a mechanism to reduce copy overheads in TCP-based messaging
IPDPS'06 Proceedings of the 20th international conference on Parallel and distributed processing
Reducing TPC-H benchmarking time
PCI'05 Proceedings of the 10th Panhellenic conference on Advances in Informatics
Accuracy of performance prediction for EJB applications: a statistical analysis
SEM'04 Proceedings of the 4th international conference on Software Engineering and Middleware
Performance prediction of J2EE applications using messaging protocols
CBSE'05 Proceedings of the 8th international conference on Component-Based Software Engineering
DFT performance prediction in FFTW
LCPC'09 Proceedings of the 22nd international conference on Languages and Compilers for Parallel Computing
Evaluating program analysis and testing tools with the RUGRAT random benchmark application generator
Proceedings of the 2012 Workshop on Dynamic Analysis
CarFast: achieving higher statement coverage faster
Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering
Estimating parallel performance
Journal of Parallel and Distributed Computing
International Journal of High Performance Computing Applications
Hi-index | 0.01 |
Standard benchmarking provides to run-times for given programs on given machines, but fails to provide insight as to why those results were obtained (either in terms of machine or program characteristics) and fails to provide run-times for that program on some other machine, or some other programs on that machine. We have developed a machine-imdependent model of program execution to characterize both machine performance and program execution. By merging these machine and program characterizations, we can estimate execution time for arbitrary machine/program combinations. Our technique allows us to identify those operations, either on the machine or in the programs, which dominate the benchmark results. This information helps designers in improving the performance of future machines and users in tuning their applications to better utilize the performance of existing machines. Here we apply our methodology to characterize benchmarks and predict their execution times. We present extensive run-time statistics for a large set of benchmarks including the SPEC and Perfect Club suites. We show how these statistics can be used to identify important shortcoming in the programs. In addition, we give execution time estimates for a large sample of programs and machines and compare these against benchmark results. Finally, we develop a metric for program similarity that makes it possible to classify benchmarks with respect to a large set of characteristics.