Human evaluation of article and noun number usage: influences of context and construction variability

  • Authors:
  • John Lee;Joel Tetreault;Martin Chodorow

  • Affiliations:
  • MIT CSAIL, Cambridge, MA;Educational Testing Service, Princeton, NJ;Hunter College of CUNY, New York, NY

  • Venue:
  • ACL-IJCNLP '09 Proceedings of the Third Linguistic Annotation Workshop
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Evaluating systems that correct errors in non-native writing is difficult because of the possibility of multiple correct answers and the variability in human agreement. This paper seeks to improve the best practice of such evaluation by analyzing the frequency of multiple correct answers and identifying factors that influence agreement levels in judging the usage of articles and noun number.