Automated extraction of tags from the penn treebank

  • Authors:
  • John Chen;Vijay K. Shanker

  • Affiliations:
  • Department of Computer Science, Columbia University, New York, NY;Department of Computer and Information Sciences, University of Delaware, Newark, DE

  • Venue:
  • New developments in parsing technology
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

The accuracy of statistical parsing models can be improved with the use of lexical information. Statistical parsing using Lexicalized tree adjoining grammar (LTAG), a kind of lexicalized grammar, has remained relatively unexplored. We believe that is largely in part due to the absence of large corpora accurately bracketed in terms of a perspicuous yet broad coverage LTAG. Our work attempts to alleviate this difficulty. We extract different LTAGs from the Penn Treebank. We show that certain strategies yield an improved extracted LTAG in terms of compactness, broad coverage, and supertagging accuracy. Furthermore, we perform a preliminary investigation in smoothing these grammars by means of an external linguistic resource, namely, the tree families of the XTAG grammar, a hand built grammar of English.