Constructing multiclass learners from binary learners: a simple black-box analysis of the generalization errors

Authors:
Jittat Fakcharoenphol;Boonserm Kijsirikul
Affiliations:
Department of Computer Engineering, Kasetsart University, Bangkok, Thailand;Department of Computer Engineering, Chulalongkorn University, Bangkok, Thailand
Venue:
ALT'05 Proceedings of the 16th international conference on Algorithmic Learning Theory
Year:
2005

Citing 16
Cited 1

Learnability and the Vapnik-Chervonenkis dimension

Journal of the ACM (JACM)
Characterizations of learnability for classes of {0, …, n}-valued functions

Journal of Computer and System Sciences
Support-Vector Networks

Machine Learning
A framework for structural risk minimisation

COLT '96 Proceedings of the ninth annual conference on Computational learning theory
A decision-theoretic generalization of on-line learning and an application to boosting

Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Pairwise classification and support vector machines

Advances in kernel methods
Classification by pairwise coupling

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Multiclass learning, boosting, and error-correcting codes

COLT '99 Proceedings of the twelfth annual conference on Computational learning theory
Enlarging the Margins in Perceptron Decision Trees

Machine Learning
On Learning Sets and Functions

Machine Learning
Adaptive Directed Acyclic Graphs for Multiclass Classification

PRICAI '02 Proceedings of the 7th Pacific Rim International Conference on Artificial Intelligence: Trends in Artificial Intelligence
Generalization Performance of Multiclass Discriminant Models

IJCNN '00 Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN'00)-Volume 4 - Volume 4
Reducing multiclass to binary: a unifying approach for margin classifiers

The Journal of Machine Learning Research
On nearest-neighbor error-correcting output codes with application to all-pairs multiclass support vector machines

The Journal of Machine Learning Research
In Defense of One-Vs-All Classification

The Journal of Machine Learning Research
Error-correcting output codes: a general method for improving multiclass inductive learning programs

AAAI'91 Proceedings of the ninth National conference on Artificial intelligence - Volume 2

Enhancing directed binary trees for multi-class classification

Information Sciences: an International Journal

Quantified Score

Hi-index	0.00

Visualization

Abstract

Multiclass learning is widely solved by reducing to a set of binary problems. By considering base binary classifiers as black boxes, we analyze generalization errors of various constructions, including Max-Win, Decision Directed Acyclic Graphs, Adaptive Directed Acyclic Graphs, and the unifying approach based on coding matrix with Hamming decoding of Allwein, Schapire, and Singer, using only elementary probabilistic tools. Many of these bounds are new, some are much simpler than previously known. This technique also yields a simple proof of the equivalences of the learnability and polynomial-learnability of the multiclass problem and the induced pairwise problems.