SIGDIAL '10 Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Toward learning and evaluation of dialogue policies with text examples
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
| Hi-index | 0.00 |
We are developing a corpus-based approach for the prediction of help-desk responses from features in customers' emails, where responses are represented at two levels of granularity: document and sentence. We present an automatic evaluation of the responses generated by our system, as well as a manual one involving human judges. The automatic evaluation involves textual comparisons between generated responses and responses composed by the help-desk operators. The results show that both levels of granularity produce good responses, addressing inquiries of different kinds. The human-based evaluation measures response informativeness, and confirms our conclusion that both levels of granularity produce useful responses.