Attitudes toward microcomputers: development and construct validation of a measure
International Journal of Man-Machine Studies
Designing the user interface (2nd ed.): strategies for effective human-computer interaction
Designing the user interface (2nd ed.): strategies for effective human-computer interaction
Interactive speech technology: human factors issues in the application of speech input/output to computers
Developing interactive speech technology
Interactive speech technology
Using Multivariate Statistics (5th Edition)
Using Multivariate Statistics (5th Edition)
Spatial metaphors for a speech-based mobile city guide service
Personal and Ubiquitous Computing
Visualisation improves the usability of voice-operated mobile phone services
International Journal of Human-Computer Studies
The PARADISE Evaluation Framework: Issues and Findings
Computational Linguistics
Flexible shortcuts: designing a new speech user interface for command execution
CHI '08 Extended Abstracts on Human Factors in Computing Systems
Predicting the quality and usability of spoken dialogue services
Speech Communication
Evaluation Methods for Multimodal Systems: A Comparison of Standardized Usability Questionnaires
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Subjective Evaluation Method for Speech-Based Uni- and Multimodal Applications
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Weighting the Coefficients in PARADISE Models to Increase Their Generalizability
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Towards a Perception-Based Evaluation Model for Spoken Dialogue Systems
PIT '08 Proceedings of the 4th IEEE tutorial and research workshop on Perception and Interactive Technologies for Speech-Based Systems: Perception in Multimodal Dialogue Systems
Multimodal system evaluation using modality efficiency and synergy metrics
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
The RavenClaw dialog management framework: Architecture and systems
Computer Speech and Language
NAACL-HLT-Dialog '07 Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies
Usability Evaluation of Multimodal Interfaces: Is the Whole the Sum of Its Parts?
Proceedings of the 13th International Conference on Human-Computer Interaction. Part II: Novel Interaction Methods and Techniques
Evaluation of a Voice-Based Internet Browser with Untrained and Trained Users
UAHCI '09 Proceedings of the 5th International Conference on Universal Access in Human-Computer Interaction. Part III: Applications and Services
Reducing working memory load in spoken dialogue systems
Interacting with Computers
An evaluation understudy for dialogue coherence models
SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
A framework for model-based evaluation of spoken dialog systems
SIGdial '08 Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue
A wizard-of-Oz system evaluation study
TSD'07 Proceedings of the 10th international conference on Text, speech and dialogue
Formal and functional assessment of the pyramid method for summary content evaluation*
Natural Language Engineering
Evaluating multimodal systems: a comparison of established questionnaires and interaction parameters
Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries
Computer Speech and Language
Socializing with Olivia, the youngest robot receptionist outside the lab
ICSR'10 Proceedings of the Second international conference on Social robotics
User experience in speech recognition of navigation devices: an assessment
Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services
Exploring user satisfaction in a tutorial dialogue system
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
SIGDIAL '11 Proceedings of the SIGDIAL 2011 Conference
COST'09 Proceedings of the Second international conference on Development of Multimodal Interfaces: active Listening and Synchrony
Voice control in smart homes using distant microphones: a VoiceXML-Based approach
AmI'11 Proceedings of the Second international conference on Ambient Intelligence
SDCTD '12 NAACL-HLT Workshop on Future Directions and Needs in the Spoken Dialog Community: Tools and Data
An end-to-end evaluation of two situated dialog systems
SIGDIAL '12 Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Relating to speech evoked car personalities
CHI '13 Extended Abstracts on Human Factors in Computing Systems
Mostly passive information delivery in a car
Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
Hi-index | 0.00 |
Applications of speech recognition are now widespread, but user-centred evaluation methods are necessary to ensure their success. Objective evaluation techniques are fairly well established, but previous subjective techniques have been unstructured and unproven. This paper reports on the first stage in the development of a questionnaire measure for the Subjective Assessment of Speech System Interfaces (SASSI). The aim of the research programme is to produce a valid, reliable and sensitive measure of users' subjective experiences with speech recognition systems. Such a technique could make an important contribution to theory and practice in the design and evaluation of speech recognition systems according to best human factors practice. A prototype questionnaire was designed, based on established measures for evaluating the usability of other kinds of user interface, and on a review of the research literature into speech system design. This consisted of 50 statements with which respondents rated their level of agreement. The questionnaire was given to users of four different speech applications, and Exploratory Factor Analysis of 214 completed questionnaires was conducted. This suggested the presence of six main factors in users' perceptions of speech systems: System Response Accuracy, Likeability, Cognitive Demand, Annoyance, Habitability and Speed. The six factors have face validity, and a reasonable level of statistical reliability. The findings form a useful theoretical and practical basis for the subjective evaluation of any speech recognition interface. However, further work is recommended, to establish the validity and sensitivity of the approach, before a final tool can be produced which warrants general use.