Reducing the overconfidence of base classifiers when combining their decisions

Authors:
Šarunas Raudys;Ray Somorjai;Richard Baumgartner
Affiliations:
Vilnius Gediminas Technical University, Vilnius, Lithuania;Institute for Biodiagnostics, National Research Council Canada, Winnipeg, MB, Canada;Institute for Biodiagnostics, National Research Council Canada, Winnipeg, MB, Canada
Venue:
MCS'03 Proceedings of the 4th international conference on Multiple classifier systems
Year:
2003

Citing 16
Cited 3

Introduction to statistical pattern recognition (2nd ed.)

Introduction to statistical pattern recognition (2nd ed.)
Original Contribution: Stacked generalization

Neural Networks
Hierarchical mixtures of experts and the EM algorithm

Neural Computation
A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals

IEEE Transactions on Pattern Analysis and Machine Intelligence
Bagging predictors

Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
On Combining Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
Statistical and neural classifiers: an integrated approach to design

Statistical and neural classifiers: an integrated approach to design
Proceedings of the Third International Workshop on Multiple Classifier Systems

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
An Experimental Comparison of Fixed and Trained Fusion Rules for Crisp Classifier Outputs

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
Multiple Classification Systems in the Context of Feature Extraction and Selection

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
Reduction of the Boasting Bias of Linear Experts

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
Analysis of Linear and Order Statistics Combiners for Fusion of Imbalanced Classifiers

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
Multiclassifier Systems: Back to the Future

MCS '02 Proceedings of the Third International Workshop on Multiple Classifier Systems
Experts' Boasting in Trainable Fusion Rules

IEEE Transactions on Pattern Analysis and Machine Intelligence
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)

Determining the optimal percent of negative examples used in training the multilayer perceptron neural networks

NN'09 Proceedings of the 10th WSEAS international conference on Neural networks
Identification of signatures in biomedical spectra using domain knowledge

Artificial Intelligence in Medicine
Using domain knowledge in the random subspace method: application to the classification of biomedical spectra

MCS'05 Proceedings of the 6th international conference on Multiple Classifier Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

When the sample size is small, the optimistically biased outputs produced by expert classifiers create serious problems for the combiner rule designer. To overcome these problems, we derive analytical expressions for bias reduction for situations when the standard Gaussian density-based quadratic classifiers serve as experts and the decisions of the base experts are aggregated by the behavior-space-knowledge (BKS) method. These reduction terms diminish the experts' overconfidence and improve the multiple classification system's generalization ability. The bias-reduction approach is compared with the standard BKS, majority voting and stacked generalization fusion rules on two real-life datasets for which the different base expert aggregates comprise the multiple classification system.