Evaluation metrics for generation
INLG '00 Proceedings of the first international conference on Natural language generation - Volume 14
Statistical ranking in tactical generation
EMNLP '06 Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing
Stochastic realisation ranking for a free word order language
ENLG '07 Proceedings of the Eleventh European Workshop on Natural Language Generation
Evaluating coverage for large symbolic NLG grammars
IJCAI'03 Proceedings of the 18th international joint conference on Artificial intelligence
Probabilistic models for disambiguation of an HPSG-based chart generator
Parsing '05 Proceedings of the Ninth International Workshop on Parsing Technology
Correlating human and automatic evaluation of a German surface realiser
ACLShort '09 Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
Underspecifying and predicting voice for surface realisation ranking
HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
| Hi-index | 0.00 |
In this paper we present a human-based evaluation of surface realisation alternatives. We examine the relative rankings of naturally occurring corpus sentences and automatically generated strings chosen by statistical models (language model, log-linear model), as well as the naturalness of the strings chosen by the log-linear model. We also investigate to what extent preceding context has an effect on choice. We show that native speakers do accept quite some variation in word order, but there are also clearly factors that make certain realisation alternatives more natural.