Examining the redundancy of multimodal input

Authors:
Natalie Ruiz;Ronnie Taib;Fang Chen
Affiliations:
Australian Technology Park, Sydney, Australia and University of New South Wales, Sydney, Australia;Australian Technology Park, Sydney, Australia and University of New South Wales, Sydney, Australia;Australian Technology Park, Sydney, Australia and University of New South Wales, Sydney, Australia
Venue:
OZCHI '06 Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments
Year:
2006

Citing 1
Cited 7

When do we interact multimodally?: cognitive load and multimodal communication patterns

Proceedings of the 6th international conference on Multimodal interfaces

Using pen input features as indices of cognitive load

Proceedings of the 9th international conference on Multimodal interfaces
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
Integrating semantics into multimodal interaction patterns

MLMI'07 Proceedings of the 4th international conference on Machine learning for multimodal interaction
Review: Integrating cognitive load theory and concepts of human-computer interaction

Computers in Human Behavior
Teaching athletes cognitive skills: detecting cognitive load in speech input

BCS '10 Proceedings of the 24th BCS Interaction Specialist Group Conference
Multimodal behavior and interaction as indicators of cognitive load

ACM Transactions on Interactive Intelligent Systems (TiiS) - Special issue on highlights of the decade in interactive intelligent systems
Analysing mouse activity for cognitive load detection

Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration

Quantified Score

Hi-index	0.00

Visualization

Abstract

Speech and gesture modalities can allow users to interact with complex applications in novel ways. Often users will adapt their multimodal behaviour to cope with increasing levels of domain complexity. These strategies can change how multimodal constructions are planned and executed by users. In the frame of Baddeley's Theory of Working Memory, we present some of the results from an empirical study conducted with users of a multimodal interface, under varying levels of cognitive load. In particular, we examine how multimodal behavioural features are sensitive to cognitive load variations. We report significant decreases in multimodal redundancy (33.6%) and trends of increased multimodal complementarity, as cognitive load increases.