An iterative process for building learning curves and predicting relative performance of classifiers

  • Authors:
  • Rui Leite;Pavel Brazdil

  • Affiliations:
  • LIAAD, INESC Porto L.A., Faculty of Economics, University of Porto, Porto, Portugal;LIAAD, INESC Porto L.A., Faculty of Economics, University of Porto, Porto, Portugal

  • Venue:
  • EPIA'07 Proceedings of the aritficial intelligence 13th Portuguese conference on Progress in artificial intelligence
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper concerns the problem of predicting the relative performance of classification algorithms. Our approach requires that experiments are conducted on small samples. The information gathered is used to identify the nearest learning curve for which the sampling procedure was fully carried out. This allows the generation of a prediction regarding the relative performance of the algorithms. The method automatically establishes how many samples are needed and their sizes. This is done iteratively by taking into account the results of all previous experiments - both on other datasets and on the new dataset obtained so far. Experimental evaluation has shown that the method achieves better performance than previous approaches.