A tool for automated evaluation of algorithms

  • Authors:

  • Philipp Kranen;Stephan Wels;Tim Rohlfs;Sebastian Raubach;Thomas Seidl

  • Affiliations:

  • RWTH Aachen University, Aachen, Germany;RWTH Aachen University, Aachen, Germany;RWTH Aachen University, Aachen, Germany;RWTH Aachen University, Aachen, Germany;RWTH Aachen University, Aachen, Germany

  • Venue:
  • Proceedings of the 21st ACM international conference on Information and knowledge management
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Testing algorithms and systems involves trying different sets of parameter values on different domains or data sets. Even for a moderate number of parameters and domains the number of possible experiments can get very large due to the combinatorial explosion. Evaluating the outcome of these experiments requires comparing the results, which is often done by writing a script or inspecting the result files manually. For a new algorithm or version, the work has to be done over again. With hundreds, thousands, or even more possible experiments, both the preparation and the evaluation can become complex and tedious. In this demonstrator we present a software tool, called ET, for evaluating the parameters of an algorithm or system, either automatically or controlled by the user. It allows to launch large numbers of experiments in just a few clicks, visually explore the results and analyze the performance of the algorithm.