Evaluating algorithms for the generation of referring expressions using a balanced corpus

  • Authors:
  • Albert Gatt;Ielka van der Sluis;Kees van Deemter

  • Affiliations:
  • University of Aberdeen;University of Aberdeen;University of Aberdeen

  • Venue:
  • ENLG '07 Proceedings of the Eleventh European Workshop on Natural Language Generation
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

Despite being the focus of intensive research, evaluation of algorithms that generate referring expressions is still in its infancy. We describe a corpus-based evaluation methodology, applied to a number of classic algorithms in this area. The methodology focuses on balance and semantic transparency to enable comparison of human and algorithmic output. Although the Incremental Algorithm emerges as the best match, we found that its dependency on manually-set parameters makes its performance difficult to predict.