Oracle coached decision trees and lists

Authors:
Ulf Johansson;Cecilia Sönströd;Tuve Löfström
Affiliations:
CSL@BS Research Group School of Business and Informatics, University of Borås, Sweden;CSL@BS Research Group School of Business and Informatics, University of Borås, Sweden;CSL@BS Research Group School of Business and Informatics, University of Borås, Sweden
Venue:
IDA'10 Proceedings of the 9th international conference on Advances in Intelligent Data Analysis
Year:
2010

Citing 8
Cited 0

C4.5: programs for machine learning

C4.5: programs for machine learning
Bagging predictors

Machine Learning
Random Forests

Machine Learning
Transductive Inference for Text Classification using Support Vector Machines

ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
Using Rule Sets to Maximize ROC Performance

ICDM '01 Proceedings of the 2001 IEEE International Conference on Data Mining
Rule extraction: using neural networks or for neural networks?

Journal of Computer Science and Technology
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper introduces a novel method for obtaining increased predictive performance from transparent models in situations where production input vectors are available when building the model. First, labeled training data is used to build a powerful opaque model, called an oracle. Second, the oracle is applied to production instances, generating predicted target values, which are used as labels. Finally, these newly labeled instances are utilized, in different combinations with normal training data, when inducing a transparent model. Experimental results, on 26 UCI data sets, show that the use of oracle coaches significantly improves predictive performance, compared to standard model induction. Most importantly, both accuracy and AUC results are robust over all combinations of opaque and transparent models evaluated. This study thus implies that the straightforward procedure of using a coaching oracle, which can be used with arbitrary classifiers, yields significantly better predictive performance at a low computational cost.