Enriching the knowledge sources used in a maximum entropy part-of-speech tagger

  • Authors:
  • Kristina Toutanova;Christopher D. Manning

  • Affiliations:
  • Stanford, CA;Stanford, CA

  • Venue:
  • EMNLP '00 Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics - Volume 13
  • Year:
  • 2000

Quantified Score

Hi-index 0.01

Visualization

Abstract

This paper presents results for a maximum-entropy-based part of speech tagger, which achieves superior performance principally by enriching the information sources used for tagging. In particular, we get improved results by incorporating these features: (i) more extensive treatment of capitalization for unknown words; (ii) features for the disambiguation of the tense forms of verbs; (iii) features for disambiguating particles from prepositions and adverbs. The best resulting accuracy for the tagger on the Penn Treebank is 96.86% overall, and 86.91% on previously unseen words.