Beyond class A: a proposal for automatic evaluation of discourse
HLT '90 Proceedings of the workshop on Speech and Natural Language
Towards developing general models of usability with PARADISE
Natural Language Engineering
PARADISE: a framework for evaluating spoken dialogue agents
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Full Bayesian network classifiers
ICML '06 Proceedings of the 23rd international conference on Machine learning
Partially observable Markov decision processes for spoken dialog systems
Computer Speech and Language
Relations between de-facto criteria in the evaluation of a spoken dialogue system
Speech Communication
Boosted Bayesian network classifiers
Machine Learning
The RavenClaw dialog management framework: Architecture and systems
Computer Speech and Language
A Bayesian NETWORKS approach for dialog modeling: The fusion BN
ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
Assessing the effects of building social intelligence in a robotic interface for the home
Interacting with Computers
New research perspectives on Ambient Intelligence
Journal of Ambient Intelligence and Smart Environments
NEMOHIFI: an affective HiFi agent
Proceedings of the 15th ACM on International conference on multimodal interaction
Hi-index | 0.00 |
In this paper a Bayesian Networks-based solution for dialogue modelling is presented. This solution is combined with carefully designed contextual information handling strategies. With the purpose of validating these solutions, and introducing a spoken dialogue system for controlling a Hi-Fi audio system as the selected prototype, a real-user evaluation has been conducted. Two different versions of the prototype are compared. Each version corresponds to a different implementation of the algorithm for the management of the actuation order, the algorithm for deciding the proper order to carry out the actions required by the user. The evaluation is carried out in terms of a battery of both subjective and objective metrics collected from speakers interacting with the Hi-Fi audio box through predefined scenarios. Defined metrics have been specifically adapted to measure: first, the usefulness and the actual relevance of the proposed solutions, and, secondly, their joint performance through their intelligent combination mainly measured as the level achieved with regard to the user satisfaction. A thorough and comprehensive study of the main differences between both approaches is presented. Two-way analysis of variance (ANOVA) tests are also included to measure the effects of both: the system used and the type of scenario factors, simultaneously. Finally, the effect of bringing this flexibility, robustness and naturalness into our home dialogue system is also analyzed through the results obtained. These results show that the intelligence of our speech interface has been well perceived, highlighting its excellent ease of use and its good acceptance by users, therefore validating the approached dialogue management solutions and demonstrating that a more natural, flexible and robust dialogue is possible thanks to them.