Over-Fitting in ensembles of neural network classifiers within ECOC frameworks

Authors:
Matthew Prior;Terry Windeatt
Affiliations:
Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford, Surrey, UK;Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford, Surrey, UK
Venue:
MCS'05 Proceedings of the 6th international conference on Multiple Classifier Systems
Year:
2005

Citing 8
Cited 3

Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Estimating Generalization Error on Two-Class Datasets Using Out-of-Bag Estimates

Machine Learning
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Machine Learning
Using Error-Correcting Codes for Text Classification

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Assessing and Improving Neural Network Predictions by the Bootstrap Algorithm

Advances in Neural Information Processing Systems 5, [NIPS Conference]
In Defense of One-Vs-All Classification

The Journal of Machine Learning Research
Solving multiclass learning problems via error-correcting output codes

Journal of Artificial Intelligence Research
A new ensemble diversity measure applied to thinning ensembles

MCS'03 Proceedings of the 4th international conference on Multiple classifier systems

Overfitting cautious selection of classifier ensembles with genetic algorithms

Information Fusion
Multiple Classifier Systems for Adversarial Classification Tasks

MCS '09 Proceedings of the 8th International Workshop on Multiple Classifier Systems
An ensemble dependence measure

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

We have investigated the performance of a generalisation error predictor, Gest, in the context of error correcting output coding ensembles based on multi-layer perceptrons. An experimental evaluation on benchmark datasets with added classification noise shows that over-fitting can be detected and a comparison is made with the Q measure of ensemble diversity. Each dichotomy associated with a column of an ECOC code matrix is presented with a bootstrap sample of the training set. Gest uses the out-of-bootstrap samples to efficiently estimate the mean column error for the independent test set and hence the test error. This estimate can then be used select a suitable complexity for the base classifiers in the ensemble.