Quantifying homogeneity of instance sets for algorithm configuration

  • Authors:
  • Marius Schneider;Holger H. Hoos

  • Affiliations:
  • University of Potsdam, Germany;Unversity of British Columbia, Canada

  • Venue:
  • LION'12 Proceedings of the 6th international conference on Learning and Intelligent Optimization
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Automated configuration procedures play an increasingly prominent role in realising the performance potential inherent in highly parametric solvers for a wide range of computationally challenging problems. However, these configuration procedures have difficulties when dealing with inhomogenous instance sets, where the relative difficulty of problem instances varies between configurations of the given parametric algorithm. In the literature, instance set homogeneity has been assessed using a qualitative, visual criterion based on heat maps. Here, we introduce two quantitative measures of homogeneity and empirically demonstrate these to be consistent with the earlier qualitative criterion. We also show that according to our measures, homogeneity increases when partitioning instance sets by means of clustering based on observed runtimes, and that the performance of a prominent automatic algorithm configurator increases on the resulting, more homogenous subsets.