Evaluating classifiers' performance in a constrained environment

• Authors: Anna Olecka
• Venue: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining
• Year: 2002

In this paper, we focus on methodology of finding a classifier with a minimal cost in presence of additional performance constraints. ROCCH analysis, where accuracy and cost are intertwined in the solution space, was a revolutionary tool for two-class problems. We propose an alternative formulation, as an optimization problem, commonly used in Operations Research. This approach extends the ROCCH analysis to allow for locating optimal solutions while outside constraints are present. Similarly to the ROCCH analysis, we combine cost and class distribution while defining the objective function. Rather than focusing on slopes of the edges in the convex hull of the solution space, however, we treat cost as an objective function to be minimized over the solution space, by selecting the best performing classifier(s) (one or more vertex in the solution space). The Linear Programming framework provides a theoretical and computational methodology for finding the vertex (classifier) which minimizes the objective function.