Task-based evaluation of NLG systems: control vs real-world context

Authors:
Ehud Reiter
Affiliations:
University of Aberdeen
Venue:
UCNLG+EVAL '11 Proceedings of the UCNLG+Eval: Language Generation and Evaluation Workshop
Year:
2011

Citing 6
Cited 0

Using Grice's maxim of quantity to select the content of plan descriptions

Artificial Intelligence
Lessons from a failure: generating tailored smoking cessation letters

Artificial Intelligence
The challenge of information visualization evaluation

Proceedings of the working conference on Advanced visual interfaces
Aggregation improves learning: experiments in natural language generation for intelligent tutoring systems

ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Automatic generation of textual summaries from neonatal intensive care data

Artificial Intelligence
From data to text in the Neonatal Intensive Care Unit: Using NLG technology for decision support and information management

AI Communications

Quantified Score

Hi-index	0.00

Visualization

Abstract

Currently there is little agreement about, or even discussion of, methodologies for task-based evaluation of NLG systems. I discuss one specific issue in this area, namely the importance of control vs the importance of ecological validity (real-world context), and suggest that perhaps we need to put more emphasis on ecological validity in NLG evaluations.