Towards developing general models of usability with PARADISE
Natural Language Engineering
PARADISE: a framework for evaluating spoken dialogue agents
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
The human-robot interaction operating system
Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction
Using Multivariate Statistics (5th Edition)
Using Multivariate Statistics (5th Edition)
Comparing objective and subjective measures of usability in a human-robot dialogue system
ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2
The curious robot-structuring interactive robot learning
ICRA'09 Proceedings of the 2009 IEEE international conference on Robotics and Automation
A regression-based approach to modeling addressee backchannels
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
| Hi-index | 0.00 |
We present the design process, realization and evaluation of a robot system for nteractive object learning. The system-oriented evaluation, in particular, addresses an open problem for the evaluation of systems, where overall user satisfaction depends not only on the performance of the parts, but also on their combination, and on user behavior. Based on the PARADISE method known from spoken dialog systems, we have defined and applied internal and external metrics for fine-grained and largely automatable identification of such relationships. Through evaluation with n=28 subjects, indicator functions explaining up to 55% of variation in several satisfaction metrics were found. Furthermore, we demonstrate that the system's interaction style reduces the need for instruction and successfully recovers partial failures.