Speech, language, and audition
Human-computer interaction
On Cognitive Relevance in Automatic Multimodal Systems
ISMSE '04 Proceedings of the IEEE Sixth International Symposium on Multimedia Software Engineering
Towards Cognitive-Aware Multimodal Presentation: The Modality Effects in High-Load HCI
EPCE '09 Proceedings of the 8th International Conference on Engineering Psychology and Cognitive Ergonomics: Held as Part of HCI International 2009
Review: Integrating cognitive load theory and concepts of human-computer interaction
Computers in Human Behavior
Computers in Human Behavior
On a NeuroIS design science model
DESRIST'11 Proceedings of the 6th international conference on Service-oriented perspectives in design science research
A reference model for adaptive visualization systems
HCII'11 Proceedings of the 14th international conference on Human-computer interaction: design and development approaches - Volume Part I
| Hi-index | 0.00 |
In this study, we argue that modality planning in multimodal presentation systems needs to consider the modality characteristics at not only the presentational level but also the cognitive level, especially in a situation where the information load is high and the user task is time-critical. As a first step towards automatic cognitive-aware modality planning, we integrated the effect of different modalities on cognitive load and performance, using a high-load information presentation scenario. Mainly based on modality-related psychology theories, we selected five modality conditions (text, image, text+image, text+speech, and text+sound) and made hypotheses about their effects on cognitive load. Modality effects were evaluated by two cognitive load measurements and two performance measurements. Results confirmed most of the predicted modality effects, and showed that these effects become significant when the information load and the task demand are high. The findings of this study suggest that it is highly necessary to encode modality-related principles of human cognition into the modality planning procedure for systems that support high-load human-computer interaction.