PARADISE: a framework for evaluating spoken dialogue agents
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Learning optimal dialogue strategies: a case study of a spoken dialogue agent for email
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
Automatic optimization of dialogue management
COLING '00 Proceedings of the 18th conference on Computational linguistics - Volume 1
Quantitative and qualitative evaluation of Darpa Communicator spoken dialogue systems
ACL '01 Proceedings of the 39th Annual Meeting on Association for Computational Linguistics
Identifying user corrections automatically in spoken dialogue systems
NAACL '01 Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies
Spoken dialogue evaluation for the Bell Labs communicator system
HLT '02 Proceedings of the second international conference on Human Language Technology Research
Correction grammars for error handling in a speech dialog system
HLT-NAACL-Short '04 Proceedings of HLT-NAACL 2004: Short Papers
Toward a computational approach for natural language description of emotions
ACII'11 Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II
| Hi-index | 0.00 |
This paper examines the use of model trees for evaluating user utterances for response to system error in dialogs from the Communicator 2000 corpus. The features used by the model trees are limited to those which can be automatically obtained through acoustic measurements. These features are derived from pitch and energy measurements. The curve of the model tree output versus dialog turn is interpreted to be a measure of the level of user activation in the dialog. We test the premise that user response to error at the utterance level is related to user satisfaction at the dialog level. Several different evaluation tasks are investigated: on an utterance level we applied the model tree output to detecting response to error and on the dialog level we analyzed the relation of model tree output to estimating user satisfaction. For the former, we achieve 65% precision and 63% recall and for the latter our predictions show significant .48 correlation with user surveys.