Statistical approaches to combining binary classifiers for multi-class classification

Authors:
Yuichi Shiraishi;Kenji Fukumizu
Affiliations:
Human Genome Center, Institute of Medical Science, The University of Tokyo, 4-6-1 Shirokanedai, Minato-ku, Tokyo 108-8639, Japan;The Institute of Statistical Mathematics, 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan
Venue:
Neurocomputing
Year:
2011

Citing 23
Cited 2

Original Contribution: Stacked generalization

Neural Networks
Stacked regressions

Machine Learning
On Combining Classifiers

IEEE Transactions on Pattern Analysis and Machine Intelligence
Fast training of support vector machines using sequential minimal optimization

Advances in kernel methods
Multicategory Classification by Support Vector Machines

Computational Optimization and Applications - Special issue on computational optimization—a tribute to Olvi Mangasarian, part I
Cascade Generalization

Machine Learning
On the Learnability and Design of Output Codes for Multiclass Problems

Machine Learning
Combining Classifiers with Meta Decision Trees

Machine Learning
Reducing multiclass to binary: a unifying approach for margin classifiers

The Journal of Machine Learning Research
On the algorithmic implementation of multiclass kernel-based vector machines

The Journal of Machine Learning Research
Is Combining Classifiers with Stacking Better than Selecting the Best One?

Machine Learning
In Defense of One-Vs-All Classification

The Journal of Machine Learning Research
Probability Estimates for Multi-class Classification by Pairwise Coupling

The Journal of Machine Learning Research
The Combination of Text Classifiers Using Reliability Indicators

Information Retrieval
Statistical Analysis of Some Multi-Category Large Margin Classification Methods

The Journal of Machine Learning Research
Statistical Comparisons of Classifiers over Multiple Data Sets

The Journal of Machine Learning Research
Generalized Bradley-Terry Models and Multi-Class Probability Estimates

The Journal of Machine Learning Research
Building Support Vector Machines with Reduced Classifier Complexity

The Journal of Machine Learning Research
Multi-class Protein Classification Using Adaptive Codes

The Journal of Machine Learning Research
On the Consistency of Multiclass Classification Methods

The Journal of Machine Learning Research
Solving multiclass learning problems via error-correcting output codes

Journal of Artificial Intelligence Research
Which is the best multiclass SVM method? an empirical study

MCS'05 Proceedings of the 6th international conference on Multiple Classifier Systems
A comparison of methods for multiclass support vector machines

IEEE Transactions on Neural Networks

Error-correcting output codes based ensemble feature extraction

Pattern Recognition
Data weighting method on the basis of binary encoded output to solve multi-class pattern classification problems

Expert Systems with Applications: An International Journal

Quantified Score

Hi-index	0.01

Visualization

Abstract

One of the popular methods for multi-class classification is to combine binary classifiers. In this paper, we propose a new approach for combining binary classifiers. Our method trains a combining method of binary classifiers using statistical techniques such as penalized logistic regression, stacking, and a sparsity promoting penalty. Our approach has several advantages. Firstly, our method outperforms existing methods even if the base classifiers are well-tuned. Secondly, an estimate of conditional probability for each class can be naturally obtained. Furthermore, we propose selecting relevant binary classifiers by adding the group lasso type penalty in training the combining method.