The quickhull algorithm for convex hulls
ACM Transactions on Mathematical Software (TOMS)
Proceedings of the sixteenth annual symposium on Computational geometry
The 16th Annual ACM Symposium on Computational Geometry
Convex hulls of finite sets of points in two and three dimensions
Communications of the ACM
Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence
Genetic Algorithms in Search, Optimization and Machine Learning
Genetic Algorithms in Search, Optimization and Machine Learning
Multi-Objective Optimization Using Evolutionary Algorithms
Multi-Objective Optimization Using Evolutionary Algorithms
Automatically characterizing large scale program behavior
Proceedings of the 10th international conference on Architectural support for programming languages and operating systems
Workload Design: Selecting Representative Program-Input Pairs
Proceedings of the 2002 International Conference on Parallel Architectures and Compilation Techniques
A Statistically Rigorous Approach for Improving Simulation Methodology
HPCA '03 Proceedings of the 9th International Symposium on High-Performance Computer Architecture
MisSPECulation: partial and misleading use of SPEC CPU2000 in computer architecture conferences
Proceedings of the 30th annual international symposium on Computer architecture
SMARTS: accelerating microarchitecture simulation via rigorous statistical sampling
Proceedings of the 30th annual international symposium on Computer architecture
The Use and Abuse of SPEC: An ISCA Panel
IEEE Micro
War of the benchmark means: time for a truce
ACM SIGARCH Computer Architecture News
TurboSMARTS: accurate microarchitecture simulation sampling in minutes
SIGMETRICS '05 Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Improving Computer Architecture Simulation Methodology by Adding Statistical Rigor
IEEE Transactions on Computers
MinneSPEC: A New SPEC Benchmark Workload for Simulation-Based Computer Architecture Research
IEEE Computer Architecture Letters
Measuring Benchmark Similarity Using Inherent Program Characteristics
IEEE Transactions on Computers
The exigency of benchmark and compiler drift: designing tomorrow's processors with yesterday's tools
Proceedings of the 20th annual international conference on Supercomputing
The harmonic or geometric mean: does it really matter?
ACM SIGARCH Computer Architecture News
Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation)
Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation)
Subsetting the SPEC CPU2006 benchmark suite
ACM SIGARCH Computer Architecture News
Analysis of redundancy and application balance in the SPEC CPU2006 benchmark suite
Proceedings of the 34th annual international symposium on Computer architecture
Measuring Program Similarity: Experiments with SPEC CPU Benchmark Suites
ISPASS '05 Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software, 2005
Improve simulation efficiency using statistical benchmark subsetting: an ImplantBench case study
Proceedings of the 45th annual Design Automation Conference
Evolutionary Benchmark Subsetting
IEEE Micro
Phase complexity surfaces: characterizing time-varying program behavior
HiPEAC'08 Proceedings of the 3rd international conference on High performance embedded architectures and compilers
Selecting representative benchmark inputs for exploring microprocessor design spaces
ACM Transactions on Architecture and Code Optimization (TACO)
| Hi-index | 0.00 |
Motivated by excessively high benchmarking efforts caused by a rapidly expanding design space, increasing system complexity, and prevailing practices based on ad-hoc and subjective schemes, this article seeks to enhance architecture exploration and evaluation efficiency by strategically integrating a genetic algorithm, 3-D geometrical rendering, and multivariate statistical analysis into one unified methodology framework—SubsetTrio—capable of subsetting any given benchmark suite based on its inherent workload characteristics, desired workload space coverage, and the total execution time intended by the user. By encoding both representativity (i.e., workload space coverage represented by the volume of the convex hull of benchmarks) and efficiency (i.e., total run time) as a co-optimization objective of a survival-of-the-fittest evolutionary algorithm, we can systematically determine a globally “fittest” (i.e., most representative and efficient) benchmark subset according to the workload space coverage threshold specified by the user. We demonstrate the usage, efficacy, and efficiency of the proposed technique by conducting a thorough case study on the SPEC benchmark suite, and evaluate its validity based on 50 commercial computer systems. Compared to the state-of-the-art statistical subsetting approach based on the Principal Component Analysis (PCA), SubsetTrio could select a significantly more time-efficient subset, while covering the same or higher workload space.