Genetic rule extraction optimizing brier score

Authors:
Ulf Johansson;Rikard König;Lars Niklasson
Affiliations:
University of Borås, Borås, Sweden;University of Borås, Borås, Sweden;University of Skövde, Skövde, Sweden
Venue:
Proceedings of the 12th annual conference on Genetic and evolutionary computation
Year:
2010

Citing 10
Cited 1

C4.5: programs for machine learning

C4.5: programs for machine learning
Extracting Refined Rules from Knowledge-Based Neural Networks

Machine Learning
Using neural networks for data mining

Future Generation Computer Systems - Special double issue on data mining
Random Forests

Machine Learning
Using Rule Sets to Maximize ROC Performance

ICDM '01 Proceedings of the 2001 IEEE International Conference on Data Mining
NeuroRule: A Connectionist Approach to Data Mining

VLDB '95 Proceedings of the 21th International Conference on Very Large Data Bases
Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)

Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems)
Discovering the Mysteries of Neural Networks

International Journal of Hybrid Intelligent Systems
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
The use of the area under the ROC curve in the evaluation of machine learning algorithms

Pattern Recognition

Modeling medical decision making by support vector machines, explaining by rules of evolutionary algorithms with feature selection

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Most highly accurate predictive modeling techniques produce opaque models. When comprehensible models are required, rule extraction is sometimes used to generate a transparent model, based on the opaque. Naturally, the extracted model should be as similar as possible to the opaque. This criterion, called fidelity, is therefore a key part of the optimization function in most rule extracting algorithms. To the best of our knowledge, all existing rule extraction algorithms targeting fidelity use 0/1 fidelity, i.e., maximize the number of identical classifications. In this paper, we suggests and evaluate a rule extraction algorithm utilizing a more informed fidelity criterion. More specifically, the novel algorithms, which is based on genetic programming, minimizes the difference in probability estimates between the extracted and the opaque models, by using the generalized Brier score as fitness function. Experimental results from 26 UCI data sets show that the suggested algorithm obtained considerably higher accuracy and significantly better AUC than both the exact same rule extraction algorithm maximizing 0/1 fidelity, and the standard tree inducer J48. Somewhat surprisingly, rule extraction using the more informed fidelity metric normally resulted in less complex models, making sure that the improved predictive performance was not achieved on the expense of comprehensibility.