Optimizing abstaining classifiers using ROC analysis

Authors:
Tadeusz Pietraszek
Affiliations:
IBM Zurich Research Laboratory, Rüschlikon, Switzerland
Venue:
ICML '05 Proceedings of the 22nd international conference on Machine learning
Year:
2005

Citing 4
Cited 6

Robust classification systems for imprecise environments

AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Approximate statistical tests for comparing supervised classification learning algorithms

Neural Computation
Data mining: practical machine learning tools and techniques with Java implementations

Data mining: practical machine learning tools and techniques with Java implementations
Delegating classifiers

ICML '04 Proceedings of the twenty-first international conference on Machine learning

On the use of ROC analysis for the optimization of abstaining classifiers

Machine Learning
Classification of intrusion detection alerts using abstaining classifiers

Intelligent Data Analysis
Ensembles of Abstaining Classifiers Based on Rule Sets

ISMIS '09 Proceedings of the 18th International Symposium on Foundations of Intelligent Systems
Meta-conformity approach to reliable classification

Intelligent Data Analysis
On the Foundations of Noise-free Selective Classification

The Journal of Machine Learning Research
The data replication method for the classification with reject option

AI Communications

Quantified Score

Hi-index	0.00

Visualization

Abstract

Classifiers that refrain from classification in certain cases can significantly reduce the misclassification cost. However, the parameters for such abstaining classifiers are often set in a rather ad-hoc manner. We propose a method to optimally build a specific type of abstaining binary classifiers using ROC analysis. These classifiers are built based on optimization criteria in the following three models: cost-based, bounded-abstention and bounded-improvement. We demonstrate the usage and applications of these models to effectively reduce misclassification cost in real classification systems. The method has been validated with a ROC building algorithm and cross-validation on 15 UCI KDD datasets.