Development of an instrument measuring user satisfaction of the human-computer interface
CHI '88 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A design space for multimodal systems: concurrent processing and data fusion
CHI '93 Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems
Measuring usability: preference vs. performance
Communications of the ACM
IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use
International Journal of Human-Computer Interaction
Aesthetics and apparent usability: empirically assessing cultural and methodological issues
Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
The human-computer interaction handbook
Guidelines for multimodal user interface design
Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Towards a tool for the Subjective Assessment of Speech System Interfaces (SASSI)
Natural Language Engineering
PARADISE: a framework for evaluating spoken dialogue agents
ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
A method to standardize usability metrics into a single score
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Empirical methods for evaluating dialog systems
SIGDIAL '01 Proceedings of the Second SIGdial Workshop on Discourse and Dialogue - Volume 16
Meta-analysis of correlations among usability measures
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Predicting the quality and usability of spoken dialogue services
Speech Communication
Evaluation Methods for Multimodal Systems: A Comparison of Standardized Usability Questionnaires
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Understanding, scoping and defining user experience: a survey approach
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
User-defined gestures for surface computing
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Multimodal Interfaces: A Survey of Principles, Models and Frameworks
Human Machine Interaction
CoWME: a general framework to evaluate cognitive workload during multimodal interaction
Proceedings of the 15th ACM on International conference on multimodal interaction
| Hi-index | 0.00 |
This paper describes the analysis of established and new questionnaires concerning their applicability for the assessment of quality aspects of multimodal systems. To this purpose, an experiment with 27 participants interacting with a a smart-home system via a voice interface, a smartphone-based interface and a multimodal interface, was conducted. Interaction parameters were assessed and related to constructs measured with these questionnaires. The results indicate that some of the questionnaires are suitable for evaluating multimodal interfaces. On the basis of correlations with interaction parameters subscales of these questionnaires can be mapped to quality aspects, such as effectiveness and efficiency. Recommendations are given how to meet two important evaluation requirements, namely which questionnaire to use for comparing two or more systems or system versions and how to identify factors or components in a system that have to be improved. This is another step forward to establish evaluation methods for multimodal systems.