Comparing Naive Bayes, Decision Trees, and SVM with AUC and Accuracy

  • Authors:
  • Jin Huang;Jingjing Lu;Charles X. Ling

  • Affiliations:
  • -;-;-

  • Venue:
  • ICDM '03 Proceedings of the Third IEEE International Conference on Data Mining
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

Predictive accuracy has often been used as the mainand often only evaluation criterion for the predictive performanceof classification or data mining algorithms. Inrecent years, the area under the ROC (Receiver OperatingCharacteristics) curve, or simply AUC, has been proposedas an alternative single-number measure for evaluating performanceof learning algorithms. In our previous work, weproved that AUC is, in general, a better measure (definedprecisely) than accuracy. Many popular data mining algorithmsshould then be re-evaluated in terms of AUC. Forexample, it is well accepted that Naive Bayes and decisiontrees are very similar in accuracy. How do they compare inAUC? Also, how does the recently developed SVM (SupportVector Machine) compare to traditional learning algorithmsin accuracy and AUC? We will answer these questions inthis paper. Our conclusions will provide important guide-linesin data mining applications on real-world datasets.