Error-correcting output codes: a general method for improving multiclass inductive learning programs

  • Authors:
  • Thomas G. Dietterich;Ghulum Bakiri

  • Affiliations:
  • Department of Computer Science, Oregon State University, Corvallis, OR;Department of Computer Science, Oregon State University, Corvallis, OR

  • Venue:
  • AAAI'91 Proceedings of the ninth National conference on Artificial intelligence - Volume 2
  • Year:
  • 1991

Quantified Score

Hi-index 0.00

Visualization

Abstract

Multiclass learning problems involve finding a definition for an unknown function f(x) whose range is a discrete set containing k 2 values (i.e., k "classes"). The definition is acquired by studying large collections of training examples of the form 〈Xi, f(Xi)〉. Existing approaches to this problem include (a) direct application of multiclass algorithms such as the decision-tree algorithms ID3 and CART, (b) application of binary concept learning algorithms to learn individual binary functions for each of the k classes, and (c) application of binary concept learning algorithms with distributed output codes such as those employed by Sejnowski and Rosenberg in the NETtalk system. This paper compares these three approaches to a new technique in which BCH error-correcting codes are employed as a distributed output representation. We show that these output representations improve the performance of ID3 on the NETtalk task and of backpropagation on an isolated-letter speech-recognition task. These results demonstrate that error-correcting output codes provide a general-purpose method for improving the performance of inductive learning programs on multiclass problems.