Fine-grained Benchmark Subsetting for System Selection

  • Authors:

  • Pablo de Oliveira Castro;Yuriy Kashnikov;Chadi Akel;Mihail Popov;William Jalby

  • Affiliations:

  • Université de Versailles Saint-Quentin-en-Yvelines, France and Exascale Computing Research, France;Exascale Computing Research, France;Exascale Computing Research, France;Exascale Computing Research, France;Université de Versailles Saint-Quentin-en-Yvelines, France and Exascale Computing Research, France

  • Venue:
  • Proceedings of Annual IEEE/ACM International Symposium on Code Generation and Optimization
  • Year:
  • 2014

Quantified Score

Hi-index 0.00

Visualization

Abstract

System selection aims at finding the best architecture for a set of programs and workloads. It traditionally requires long running benchmarks. We propose a method to reduce the cost of system selection. We break down benchmarks into elementary fragments of source code, called codelets. Then, we identify two causes of redundancy: first, similar codelets; second, codelets called repeatedly. The key idea is to minimize redundancy inside the benchmark suite to speed it up. For each group of similar codelets, only one representative is kept. For codelets called repeatedly and for which the performance does not vary across calls, the number of invocations is reduced. Given an initial benchmark suite, our method produces a set of reduced benchmarks that can be used in place of the original one for system selection. We evaluate our method on the NAS SER benchmarks, producing a reduced benchmark suite 30 times faster in average than the original suite, with a maximum of 44 times. The reduced suite predicts the execution time on three target architectures with a median error between 3.9% and 8%.