Theoretical Bounds of Majority Voting Performance for a Binary Classification Problem

Authors:
Anand Narasimhamurthy
Affiliations:
IEEE
Venue:
IEEE Transactions on Pattern Analysis and Machine Intelligence
Year:
2005

Citing 10
Cited 7

Decision Combination in Multiple Classifier Systems

IEEE Transactions on Pattern Analysis and Machine Intelligence
Bagging predictors

Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
On Combining Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Theoretical Study on Six Classifier Fusion Strategies

IEEE Transactions on Pattern Analysis and Machine Intelligence
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Machine Learning
Diversity versus Quality in Classification Ensembles Based on Feature Selection

ECML '00 Proceedings of the 11th European Conference on Machine Learning
A framework for the analysis of majority voting

SCIA'03 Proceedings of the 13th Scandinavian conference on Image analysis
Application of majority voting to pattern recognition: an analysis of its behavior and performance

IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans

Using genetic programming to classify node positive patients in bladder cancer

Proceedings of the 8th annual conference on Genetic and evolutionary computation
An experimental comparison of performance measures for classification

Pattern Recognition Letters
Automatic fruit and vegetable classification from images

Computers and Electronics in Agriculture
Indexing ICD-9 codes for free-textual clinical diagnosis records by a new ensemble classifier

International Journal of Computational Intelligence in Bioinformatics and Systems Biology
On the diversity-performance relationship for majority voting in classifier ensembles

MCS'07 Proceedings of the 7th international conference on Multiple classifier systems
Information theoretic combination of pattern classifiers

Pattern Recognition
How large should ensembles of classifiers be?

Pattern Recognition

Quantified Score

Hi-index	0.14

Visualization

Abstract

A number of earlier studies that have attempted a theoretical analysis of majority voting assume independence of the classifiers. We formulate the majority voting problem as an optimization problem with linear constraints. No assumptions on the independence of classifiers are made. For a binary classification problem, given the accuracies of the classifiers in the team, the theoretical upper and lower bounds for performance obtained by combining them through majority voting are shown to be solutions of the corresponding optimization problem. The objective function of the optimization problem is nonlinear in the case of an even number of classifiers when rejection is allowed, for the other cases the objective function is linear and hence the problem is a linear program (LP). Using the framework we provide some insights and investigate the relationship between two candidate classifier diversity measures and majority voting performance.