On Evaluating Performance of Classifiers for Rare Classes

  • Authors:
  • Mahesh V. Joshi

  • Affiliations:
  • -

  • Venue:
  • ICDM '02 Proceedings of the 2002 IEEE International Conference on Data Mining
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

Predicting rare classes effectively is an important problem.The definition of effective classifier, embodied in theclassifier evaluation metric, is however very subjective, dependenton the application domain. In this paper, a widevariety of point-metrics are put into a common analyticalcontext defined by the recall and precision of the target rareclass. This enables us to compare various metrics in an objective,domain-independent manner. We judge their suitabilityfor the rare class problems along the dimensions oflearning difficulty and levels of rarity. This yields manyvaluable insights. In order to address the goal of achievingbetter recall and precision, we also propose a way ofcomparing classifiers directly based on the relationships betweenrecall and precision values. It resorts to a compositepoint-metric only when recall-precision based comparisonsyield conflicting results.