Designing and Evaluating an Adaptive Spoken Dialogue System
User Modeling and User-Adapted Interaction
Computational Linguistics
Towards a tool for the Subjective Assessment of Speech System Interfaces (SASSI)
Natural Language Engineering
Towards developing general models of usability with PARADISE
Natural Language Engineering
PARADISE: a framework for evaluating spoken dialogue agents
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
The kappa statistic: a second look
Computational Linguistics
HLT '91 Proceedings of the workshop on Speech and Natural Language
Using Multivariate Statistics (5th Edition)
Using Multivariate Statistics (5th Edition)
Humour, Relationship Maintenance and Personality Matching in automated dialogue: A controlled study
Interacting with Computers
Weighting the Coefficients in PARADISE Models to Increase Their Generalizability
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Comparing Linguistic Features for Modeling Learning in Computer Tutoring
Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work
ADiEU: Toward Domain-Based Evaluation of Spoken Dialog Systems
Proceedings of the 13th International Conference on Human-Computer Interaction. Part I: New Trends
Spoken Dialogue Interfaces: Integrating Usability
USAB '09 Proceedings of the 5th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society on HCI and Usability for e-Inclusion
A wizard-of-Oz system evaluation study
TSD'07 Proceedings of the 10th international conference on Text, speech and dialogue
Formal and functional assessment of the pyramid method for summary content evaluation*
Natural Language Engineering
Dialogue systems for virtual environments
YIWCALA '10 Proceedings of the NAACL HLT 2010 Young Investigators Workshop on Computational Approaches to Languages of the Americas
Classifying dialogue in high-dimensional space
ACM Transactions on Speech and Language Processing (TSLP)
Which system differences matter?: using l1/l2 regularization to compare dialogue systems
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
Modeling spoken dialog systems under the interactive pattern recognition framework
SSPR'12/SPR'12 Proceedings of the 2012 Joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition
Hi-index | 0.00 |
There has been a great deal of interest over the past 20 years in developing metrics and frameworks for evaluating and comparing the performance of spoken-language dialogue systems. One of the results of this interest is a potential general methodology, known as the PARADISE framework. This squib highlights some important issues concerning the application of PARADISE that have, up to now, not been sufficiently emphasized or have even been neglected by the dialogue-system community. These include considerations regarding the selection of appropriate regression parameters, normalization effects on the accuracy of the prediction, the influence of speech-recognition errors on the performance function, and the selection of an appropriate user-satisfaction measure. In addition, it gives the results of an evaluation of data from two Wizard-of-Oz experiments. These evaluations include different dependent variables and examination of individual user-satisfaction measures.