Evaluating a statistical CCG parser on Wikipedia

  • Authors:
  • Matthew Honnibal;Joel Nothman;James R. Curran

  • Affiliations:
  • University of Sydney, NSW, Australia;University of Sydney, NSW, Australia;University of Sydney, NSW, Australia

  • Venue:
  • People's Web '09 Proceedings of the 2009 Workshop on The People's Web Meets NLP: Collaboratively Constructed Semantic Resources
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

The vast majority of parser evaluation is conducted on the 1984 Wall Street Journal (WSJ). In-domain evaluation of this kind is important for system development, but gives little indication about how the parser will perform on many practical problems. Wikipedia is an interesting domain for parsing that has so far been under-explored. We present statistical parsing results that for the first time provide information about what sort of performance a user parsing Wikipedia text can expect. We find that the C&C parser's standard model is 4.3% less accurate on Wikipedia text, but that a simple self-training exercise reduces the gap to 3.8%. The self-training also speeds up the parser on newswire text by 20%.