Beyond accuracy, f-score and ROC: a family of discriminant measures for performance evaluation

  • Authors:
  • Marina Sokolova;Nathalie Japkowicz;Stan Szpakowicz

  • Affiliations:
  • DIRO, Université de Montréal, Montreal, Canada;SITE, University of Ottawa, Ottawa, Canada;SITE, University of Ottawa, Ottawa, Canada, ICS, Polish Academy of Sciences, Warsaw, Poland

  • Venue:
  • AI'06 Proceedings of the 19th Australian joint conference on Artificial Intelligence: advances in Artificial Intelligence
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Different evaluation measures assess different characteristics of machine learning algorithms. The empirical evaluation of algorithms and classifiers is a matter of on-going debate among researchers. Most measures in use today focus on a classifier's ability to identify classes correctly. We note other useful properties, such as failure avoidance or class discrimination, and we suggest measures to evaluate such properties. These measures – Youden's index, likelihood, Discriminant power – are used in medical diagnosis. We show that they are interrelated, and we apply them to a case study from the field of electronic negotiations. We also list other learning problems which may benefit from the application of these measures.