Coaxing confidences from an old friend: probabilistic classifications from transformation rule lists

  • Authors:

  • Radu Florian;John C. Henderson;Grace Ngai

  • Affiliations:

  • Johns Hopkins University, Baltimore, MD;The MITRE Corporation, Bedford, MA;Johns Hopkins University, Baltimore, MD

  • Venue:
  • EMNLP '00 Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics - Volume 13
  • Year:
  • 2000

Quantified Score

Hi-index 0.00

Visualization

Abstract

Transformation-based learning has been successfully employed to solve many natural language processing problems. It has many positive features, but one drawback is that it does not provide estimates of class membership probabilities.In this paper, we present a novel method for obtaining class membership probabilities from a transformation-based rule list classifier. Three experiments are presented which measure the modeling accuracy and cross-entropy of the probabilistic classifier on unseen data and the degree to which the output probabilities from the classifier can be used to estimate confidences in its classification decisions.The results of these experiments show that, for the task of text chunking, the estimates produced by this technique are more informative than those generated by a state-of-the-art decision tree.