Learner excellence biased by data set selection: A case for data characterisation and artificial data sets

  • Authors:
  • NúRia Macií;Ester Bernadó-Mansilla;Albert Orriols-Puig;Tin Kam Ho

  • Affiliations:
  • Grup de Recerca en Sistemes Intelligents, La Salle - Universitat Ramon Llull, C/ Quatre Camins, 2, 08022 Barcelona, Spain;Grup de Recerca en Sistemes Intelligents, La Salle - Universitat Ramon Llull, C/ Quatre Camins, 2, 08022 Barcelona, Spain;Grup de Recerca en Sistemes Intelligents, La Salle - Universitat Ramon Llull, C/ Quatre Camins, 2, 08022 Barcelona, Spain;Bell Laboratories, Alcatel-Lucent, 600 Mountain Ave., Murray Hill, NJ 07974-0636, USA

  • Venue:
  • Pattern Recognition
  • Year:
  • 2013

Quantified Score

Hi-index 0.01

Visualization

Abstract

The excellence of a given learner is usually claimed through a performance comparison with other learners over a collection of data sets. Too often, researchers are not aware of the impact of their data selection on the results. Their test beds are small, and the selection of the data sets is not supported by any previous data analysis. Conclusions drawn on such test beds cannot be generalised, because particular data characteristics may favour certain learners unnoticeably. This work raises these issues and proposes the characterisation of data sets using complexity measures, which can be helpful for both guiding experimental design and explaining the behaviour of learners.