Classifier evaluation under limited resources

Authors:
Reuven Arbel;Lior Rokach
Affiliations:
Department of Industrial Engineering, Tel-Aviv University, Israel;Department of Information Systems Engineering, Ben-Gurion University of the Negev, 84105 Beer-Sheva, Israel
Venue:
Pattern Recognition Letters
Year:
2006

Citing 5
Cited 4

Noisy replication in skewed binary classification

Computational Statistics & Data Analysis
Robust Classification for Imprecise Environments

Machine Learning
The Case against Accuracy Estimation for Comparing Induction Algorithms

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Comparisons of Classification Methods for Screening Potential Compounds

ICDM '01 Proceedings of the 2001 IEEE International Conference on Data Mining
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition

Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography

Computational Statistics & Data Analysis
Ensemble-based classifiers

Artificial Intelligence Review
Proposal for a unified methodology for evaluating supervised and non-supervised classification algorithms

CIARP'06 Proceedings of the 11th Iberoamerican conference on Progress in Pattern Recognition, Image Analysis and Applications
Predicting high-risk program modules by selecting the right software measurements

Software Quality Control

Quantified Score

Hi-index	0.10

Visualization

Abstract

Existing evaluations measures are insufficient when probabilistic classifiers are used for choosing objects to be included in a limited quota. This paper reviews performance measures that suit probabilistic classification and introduce two novel performance measures that can be used effectively for this task. It then investigates when to use each of the measures and what purpose each one of them serves. The use of these measures is demonstrated on a real life dataset obtained from the human resource field and is validated on set of benchmark datasets.