International Journal of Human-Computer Studies - Special issue on collaboration, cooperation and conflict in dialogue systems
Evaluating Natural Language Processing Systems: An Analysis and Review
Evaluating Natural Language Processing Systems: An Analysis and Review
Lessons from a failure: generating tailored smoking cessation letters
Artificial Intelligence
BLEU: a method for automatic evaluation of machine translation
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Robust PCFG-based generation using automatically acquired LFG approximations
ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
The roles of haptic-ostensive referring expressions in cooperative, task-based human-robot dialogue
Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
Intrinsic vs. extrinsic evaluation measures for referring expression generation
HLT-Short '08 Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers
A hearer-oriented evaluation of referring expression generation
ENLG '09 Proceedings of the 12th European Workshop on Natural Language Generation
A Japanese corpus of referring expressions used in a situated collaboration task
ENLG '09 Proceedings of the 12th European Workshop on Natural Language Generation
Report on the first NLG Challenge on Generating Instructions in Virtual Environments (GIVE)
ENLG '09 Proceedings of the 12th European Workshop on Natural Language Generation
The TUNA-REG Challenge 2009: overview and evaluation results
ENLG '09 Proceedings of the 12th European Workshop on Natural Language Generation
Learning content selection rules for generating object descriptions in dialogue
Journal of Artificial Intelligence Research
Choosing words in computer-generated weather forecasts
Artificial Intelligence - Special volume on connecting language to the world
Evaluating description and reference strategies in a cooperative human-robot dialogue system
IJCAI'09 Proceedings of the 21st international jont conference on Artifical intelligence
Validating the web-based evaluation of NLG systems
ACLShort '09 Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
Overspecified reference in hierarchical domains: measuring the benefits for readers
INLG '06 Proceedings of the Fourth International Natural Language Generation Conference
Noun phrase generation for situated dialogs
INLG '06 Proceedings of the Fourth International Natural Language Generation Conference
INLG '08 Proceedings of the Fifth International Natural Language Generation Conference
Evaluating evaluation methods for generation in the presence of variation
CICLing'05 Proceedings of the 6th international conference on Computational Linguistics and Intelligent Text Processing
Utilizing visual attention for cross-modal coreference interpretation
CONTEXT'05 Proceedings of the 5th international conference on Modeling and Using Context
A task-performance evaluation of referring expressions in situated collaborative task dialogues
Language Resources and Evaluation
Hi-index | 0.00 |
In the field of referring expression generation, while in the static domain both intrinsic and extrinsic evaluations have been considered, extrinsic evaluation in the dynamic domain, such as in a situated collaborative dialog, has not been discussed in depth. In a dynamic domain, a crucial problem is that referring expressions do not make sense without an appropriate preceding dialog context. It is unrealistic for an evaluation to simply show a human evaluator the whole period from the beginning of a dialog up to the time point at which a referring expression is used. Hence, to make evaluation feasible it is indispensable to determine an appropriate shorter context. In order to investigate the context necessary to understand a referring expression in a situated collaborative dialog, we carried out an experiment with 33 evaluators and a Japanese referring expression corpus. The results contribute to finding the proper contexts for extrinsic evalution in dynamic domains.