Just do what i tell you: the limited impact of instructions on multimodal integration patterns

Authors:
Sharon Oviatt;Rachel Coulston;Rebecca Lunsford
Affiliations:
Center for Human-Computer Communication, Department of Computer Science & Engineering, Oregon Health & Science University, Beaverton, OR;Center for Human-Computer Communication, Department of Computer Science & Engineering, Oregon Health & Science University, Beaverton, OR;Center for Human-Computer Communication, Department of Computer Science & Engineering, Oregon Health & Science University, Beaverton, OR
Venue:
UM'05 Proceedings of the 10th international conference on User Modeling
Year:
2005

Citing 5
Cited 2

QuickSet: multimodal interaction for distributed applications

MULTIMEDIA '97 Proceedings of the fifth ACM international conference on Multimedia
Designing computer systems for older adults

The human-computer interaction handbook
Toward a theory of organized multimodal integration patterns during human-computer interaction

Proceedings of the 5th international conference on Multimodal interfaces
Modeling multimodal integration patterns and performance in seniors: toward adaptive processing of individual differences

Proceedings of the 5th international conference on Multimodal interfaces
Individual differences in multimodal integration patterns: what are they and why do they exist?

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Toward adaptive information fusion in multimodal systems

MMUI '05 Proceedings of the 2005 NICTA-HCSNet Multimodal User Interaction Workshop - Volume 57
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

Quantified Score

Hi-index	0.00

Visualization

Abstract

Large individual differences have been documented among users in their multimodal integration patterns, which suggest that new user-adaptive approaches to multimodal fusion may be opportune. Before pursuing such an approach, this study explores whether people can be successfully encouraged to switch their multimodal integration pattern to one that is easier to process through the use of explicit instructions. Longitudinal data were collected from young and elderly adults as they used speech and pen input with a simulated map system. Results revealed that only 37% of users switched their integration pattern and maintained it, whereas another 19% never switched their natural pattern and 31% switched but then reverted during a follow-up session. In addition, significant destabilization of elderly users' integration pattern was one “cost” of attempting to instruct a change in pattern. This research underscores the need for user-centered design in future multimodal system development, especially for vulnerable users such as the elderly.