Performance testing of combinatorial solvers with isomorph class instances

  • Authors:
  • Franc Brglez;Jason A. Osborne

  • Affiliations:
  • Dept. of Computer Science, NC State University, Raleigh, NC;Dept. of Statistics, NC State University, Raleigh, NC

  • Venue:
  • ecs'07 Experimental computer science on Experimental computer science
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Combinatorial optimization problems that may be expressed as 'Boolean constraint satisfaction problems' (BCSPs) are being solved by different communities under different formulations and in different formats. If results of experimentation are reported, these can be seldom compared and replicated. We propose a pragmatic approach to reconcile these issues: (1) use the familiar LP model that naturally expresses the constraints as well as the goals of the optimization task to formulate an optimization instance, (2) assemble and translate a number of hard-to-solve instances from different domains into the .lpx format parsed by at least two BCSP solvers: lp_solve in public domain, and cplex, (3) expose the intrinsic variability of BCSP solvers by constructing instance isomorphs as an equivalence class of randomized replicas of a reference instance; (4) use isomorph classes for the design of reproducible experiments with BCSP solvers that includes performance testing hypotheses; (5) release (on the web) all data sets, reported results, and software utilities used to prepare the data, invoke experiments, and post-process the results.