Seeing the Forest Through the Trees: Learning a Comprehensible Model from an Ensemble

Authors:
Anneleen Assche;Hendrik Blockeel
Affiliations:
Computer Science Department, Katholieke Universiteit Leuven, Belgium;Computer Science Department, Katholieke Universiteit Leuven, Belgium
Venue:
ECML '07 Proceedings of the 18th European conference on Machine Learning
Year:
2007

Citing 15
Cited 1

Technical Note: Bias and the Quantification of Stability

Machine Learning - Special issue on bias evaluation and selection
Bagging predictors

Machine Learning
Random Forests

Machine Learning
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Induction of Decision Trees

Machine Learning
From Ensemble Methods to Comprehensible Models

DS '02 Proceedings of the 5th International Conference on Discovery Science
Relational Learning Using Constrained Confidence-Rated Boosting

ILP '01 Proceedings of the 11th International Conference on Inductive Logic Programming
Ensemble Methods in Machine Learning

MCS '00 Proceedings of the First International Workshop on Multiple Classifier Systems
Boosting First-Order Learning

ALT '96 Proceedings of the 7th International Workshop on Algorithmic Learning Theory
Extracting symbolic rules from trained neural network ensembles

AI Communications - Special issue on Artificial intelligence advances in China
Extracting comprehensible models from trained neural networks

Extracting comprehensible models from trained neural networks
First order random forests: Learning relational classifiers with complex aggregates

Machine Learning
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
An empirical evaluation of bagging in inductive logic programming

ILP'02 Proceedings of the 12th international conference on Inductive logic programming
Seeing the forest through the trees: learning a comprehensible model from a first order ensemble

ILP'07 Proceedings of the 17th international conference on Inductive logic programming

Comprehensible classification models: a position paper

ACM SIGKDD Explorations Newsletter

Quantified Score

Hi-index	0.00

Visualization

Abstract

Ensemble methods are popular learning methods that usually increase the predictive accuracy of a classifier though at the cost of interpretability and insight in the decision process. In this paper we aim to overcome this issue of comprehensibility by learning a single decision tree that approximates an ensemble of decision trees. The new model is obtained by exploiting the class distributions predicted by the ensemble. These are employed to compute heuristics for deciding which tests are to be used in the new tree. As such we acquire a model that is able to give insight in the decision process, while being more accurate than the single model directly learned on the data. The proposed method is experimentally evaluated on a large number of UCI data sets, and compared to an existing approach that makes use of artificially generated data.