The ROC manifold for classification systems

  • Authors:

  • Christine M. Schubert;Steven N. Thorsen;Mark E. Oxley

  • Affiliations:

  • Department of Mathematics and Statistics, Graduate School of Engineering and Management, Air Force Institute of Technology, 2950 Hobson Way, Wright-Patterson AFB, OH 45433-7765, United States;Department of Mathematical Sciences, United States Air Force Academy, 2354 Fairchild Drive, Colorado Springs, CO 80840, United States;Department of Mathematics and Statistics, Graduate School of Engineering and Management, Air Force Institute of Technology, 2950 Hobson Way, Wright-Patterson AFB, OH 45433-7765, United States

  • Venue:
  • Pattern Recognition
  • Year:
  • 2011

Quantified Score

Hi-index 0.01

Visualization

Abstract

We define the ROC manifold and CC manifold as duals in a given sense. Their analysis is required to describe the classification system. We propose a mathematical definition based on vector space methods to describe both. The ROC manifolds for n-class classification systems fully describe each system in terms of its misclassifications and, by conjunction, its correct classifications. Optimal points which minimize misclassifications can be identified even when costs and prior probabilities differ. These manifolds can be used to determine the usefulness of a classification system based on a given performance criterion. Many performance functionals (such as summary statistics) preferred for CC manifolds can also be evaluated using the ROC manifold (under certain constraints). Examples using the ROC manifold and performance functionals to compete classification systems are demonstrated with simulated and applied disease detection data.