Seeing the forest through the trees: learning a comprehensible model from a first order ensemble

Authors:
Anneleen Van Assche;Hendrik Blockeel
Affiliations:
Computer Science Department, Katholieke Universiteit Leuven, Belgium;Computer Science Department, Katholieke Universiteit Leuven, Belgium
Venue:
ILP'07 Proceedings of the 17th international conference on Inductive logic programming
Year:
2007

Citing 13
Cited 1

C4.5: programs for machine learning

C4.5: programs for machine learning
Bagging predictors

Machine Learning
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Induction of Decision Trees

Machine Learning
From Ensemble Methods to Comprehensible Models

DS '02 Proceedings of the 5th International Conference on Discovery Science
Carcinogenesis Predictions Using ILP

ILP '97 Proceedings of the 7th International Workshop on Inductive Logic Programming
Boosting First-Order Learning

ALT '96 Proceedings of the 7th International Workshop on Algorithmic Learning Theory
Extracting symbolic rules from trained neural network ensembles

AI Communications - Special issue on Artificial intelligence advances in China
Extracting comprehensible models from trained neural networks

Extracting comprehensible models from trained neural networks
First order random forests: Learning relational classifiers with complex aggregates

Machine Learning
Improving the efficiency of inductive logic programming through the use of query packs

Journal of Artificial Intelligence Research
Top-down induction of first-order logical decision trees

Artificial Intelligence
An empirical evaluation of bagging in inductive logic programming

ILP'02 Proceedings of the 12th international conference on Inductive logic programming

Seeing the Forest Through the Trees: Learning a Comprehensible Model from an Ensemble

ECML '07 Proceedings of the 18th European conference on Machine Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

Ensemble methods are popular learning methods that are usually able to increase the predictive accuracy of a classifier. On the other hand, this comes at the cost of interpretability, and insight in the decision process of an ensemble is hard to obtain. This is a major reason why ensemble methods have not been extensively used in the setting of inductive logic programming. In this paper we aim to overcome this issue of comprehensibility by learning a single first order interpretable model that approximates the first order ensemble. The new model is obtained by exploiting the class distributions predicted by the ensemble. These are employed to compute heuristics for deciding which tests are to be used in the new model. As such we obtain a model that is able to give insight in the decision process of the ensemble, while being more accurate than the single model directly learned on the data.