Active Learning to Maximize Area Under the ROC Curve

Authors:
Matt Culver;Deng Kun;Stephen Scott
Affiliations:
University of Nebraska, USA;University of Nebraska, USA;University of Nebraska, USA
Venue:
ICDM '06 Proceedings of the Sixth International Conference on Data Mining
Year:
2006

Citing 0
Cited 4

Active learning from stream data using optimal weight classifier ensemble

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics
New algorithms for budgeted learning

Machine Learning
Let us know your decision: Pool-based active training of a generative classifier with the selection strategy 4DS

Information Sciences: an International Journal
ROC analysis of classifiers in machine learning: A survey

Intelligent Data Analysis

Quantified Score

Hi-index	0.00

Visualization

Abstract

In active learning, a machine learning algorithmis given an unlabeled set of examples U, and is allowed to request labels for a relatively small subset of U to use for training. The goal is then to judiciously choose which examples in U to have labeled in order to optimize some performance criterion, e.g. classification accuracy. We study how active learning affects AUC. We examine two existing algorithms from the literature and present our own active learning algorithms designed to maximize the AUC of the hypothesis. One of our algorithms was consistently the top performer, and Closest Sampling from the literature often came in second behind it. When good posterior probability estimates were available, our heuristics were by far the best.