Mutual disambiguation of recognition errors in a multimodel architecture
Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Ten myths of multimodal interaction
Communications of the ACM
Multimodal system processing in mobile environments
UIST '00 Proceedings of the 13th annual ACM symposium on User interface software and technology
Adaptive interfaces and agents
The human-computer interaction handbook
Toward a theory of organized multimodal integration patterns during human-computer interaction
Proceedings of the 5th international conference on Multimodal interfaces
Proceedings of the 5th international conference on Multimodal interfaces
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Foreground and background interaction with sensor-enhanced mobile devices
ACM Transactions on Computer-Human Interaction (TOCHI)
Multimodal interactive maps: designing for human performance
Human-Computer Interaction
Multimodal interaction for pedestrians: an evaluation study
Proceedings of the 10th international conference on Intelligent user interfaces
Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
A study of manual gesture-based selection for the PEMMI multimodal transport management interface
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
A conceptual framework for developing adaptive multimodal applications
Proceedings of the 11th international conference on Intelligent user interfaces
Toward adaptive information fusion in multimodal systems
MMUI '05 Proceedings of the 2005 NICTA-HCSNet Multimodal User Interaction Workshop - Volume 57
Human-centered collaborative interaction
Proceedings of the 1st ACM international workshop on Human-centered multimedia
Human-centered design meets cognitive load theory: designing interfaces that help people think
MULTIMEDIA '06 Proceedings of the 14th annual ACM international conference on Multimedia
Walk to here: a voice driven animation system
Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation
Examining the redundancy of multimodal input
OZCHI '06 Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments
Signal Processing - Special section: Multimodal human-computer interfaces
TAPRAV: An interactive analysis tool for exploring workload aligned to models of task execution
Interacting with Computers
Galvanic skin response (GSR) as an index of cognitive load
CHI '07 Extended Abstracts on Human Factors in Computing Systems
Using machine learning to explore human multimodal clarification strategies
COLING-ACL '06 Proceedings of the COLING/ACL on Main conference poster sessions
Using pen input features as indices of cognitive load
Proceedings of the 9th international conference on Multimodal interfaces
Multimodal interaction analysis in a smart house
Proceedings of the 9th international conference on Multimodal interfaces
Evaluation of haptically augmented touchscreen gui elements under cognitive load
Proceedings of the 9th international conference on Multimodal interfaces
Automatic cognitive load detection from speech features
OZCHI '07 Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces
Efficiency of speech recognition for using interface design environments by novel designers
AIC'07 Proceedings of the 7th Conference on 7th WSEAS International Conference on Applied Informatics and Communications - Volume 7
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
EMU in the Car: Evaluating Multimodal Usability of a Satellite Navigation System
Interactive Systems. Design, Specification, and Verification
An integrative recognition method for speech and gestures
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
A Wizard of Oz study for an AR multimodal interface
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Designing and evaluating multimodal interaction for mobile contexts
ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Exploring cues and rhythm for designing multimodal tools to support mobile users in wayfinding
CHI '09 Extended Abstracts on Human Factors in Computing Systems
IHM '07 Proceedings of the 19th International Conference of the Association Francophone d'Interaction Homme-Machine
Expressive pen-based interfaces for math education
CSCL'07 Proceedings of the 8th iternational conference on Computer supported collaborative learning
Modeling Multimodal Interaction for Performance Evaluation
Proceedings of the 13th International Conference on Human-Computer Interaction. Part II: Novel Interaction Methods and Techniques
Exploring Multimodal Navigation Aids for Mobile Users
INTERACT '09 Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part I
Design of communication in multimodal web interfaces
Proceedings of the 27th ACM international conference on Design of communication
Auditory display design-An investigation of a design pattern approach
International Journal of Human-Computer Studies
IEEE Transactions on Audio, Speech, and Language Processing - Special issue on multimodal processing in speech-based interactions
Learning human multimodal dialogue strategies
Natural Language Engineering
HCI'07 Proceedings of the 12th international conference on Human-computer interaction: intelligent multimodal interaction environments
Evaluating usability improvements by combining visual and audio modalities in the interface
HCI'07 Proceedings of the 12th international conference on Human-computer interaction: interaction design and usability
Towards automatic cognitive load measurement from speech analysis
HCI'07 Proceedings of the 12th international conference on Human-computer interaction: interaction design and usability
Investigating effective ECAs: an experiment on modality and initiative
INTERACT'07 Proceedings of the 11th IFIP TC 13 international conference on Human-computer interaction - Volume Part II
Integrating semantics into multimodal interaction patterns
MLMI'07 Proceedings of the 4th international conference on Machine learning for multimodal interaction
Influence of feedback on the quality of multimodal systems
Proceedings of the 12th international conference on Human computer interaction with mobile devices and services
Review: Integrating cognitive load theory and concepts of human-computer interaction
Computers in Human Behavior
Using automatic techniques to evaluate usability of web multimodal interfaces design
WebMedia '09 Proceedings of the XV Brazilian Symposium on Multimedia and the Web
Improving multimodal interaction design with the MMWA authoring environment
Proceedings of the 28th ACM International Conference on Design of Communication
Cognitive skills learning: pen input patterns in computer-based athlete training
International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
Freeform pen-input as evidence of cognitive load and expertise
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Temporal binding of multimodal controls for dynamic map displays: a systems approach
ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Collaborative code reviews on interactive surfaces
Proceedings of the 29th Annual European Conference on Cognitive Ergonomics
A multi-modal interface for road planning tasks using vision, haptics and sound
ISVC'06 Proceedings of the Second international conference on Advances in Visual Computing - Volume Part II
Toward adaptive information fusion in multimodal systems
MLMI'05 Proceedings of the Second international conference on Machine Learning for Multimodal Interaction
People with motor and mobility impairment: innovative multimodal interfaces to wheelchairs
ICCHP'06 Proceedings of the 10th international conference on Computers Helping People with Special Needs
Behaviors analysis with a game-like learning system for pre-school children
ITS'06 Proceedings of the 8th international conference on Intelligent Tutoring Systems
Combining user modeling and machine learning to predict users' multimodal integration patterns
MLMI'06 Proceedings of the Third international conference on Machine Learning for Multimodal Interaction
Multimodal behavior and interaction as indicators of cognitive load
ACM Transactions on Interactive Intelligent Systems (TiiS) - Special issue on highlights of the decade in interactive intelligent systems
Assessing recovery from cognitive load through pen input
CHI '13 Extended Abstracts on Human Factors in Computing Systems
Review Article: Multimodal interaction: A review
Pattern Recognition Letters
Hi-index | 0.00 |
Mobile usage patterns often entail high and fluctuating levels of difficulty as well as dual tasking. One major theme explored in this research is whether a flexible multimodal interface supports users in managing cognitive load. Findings from this study reveal that multimodal interface users spontaneously respond to dynamic changes in their own cognitive load by shifting to multimodal communication as load increases with task difficulty and communicative complexity. Given a flexible multimodal interface, users' ratio of multimodal (versus unimodal) interaction increased substantially from 18.6% when referring to established dialogue context to 77.1% when required to establish a new context, a +315% relative increase. Likewise, the ratio of users' multimodal interaction increased significantly as the tasks became more difficult, from 59.2% during low difficulty tasks, to 65.5% at moderate difficulty, 68.2% at high and 75.0% at very high difficulty, an overall relative increase of +27%. Analysis of users' task-critical errors and response latencies across task difficulty levels increased systematically and significantly as well, corroborating the manipulation of cognitive processing load. The adaptations seen in this study reflect users' efforts to self-manage limitations on working memory when task complexity increases. This is accomplished by distributing communicative information across multiple modalities, which is compatible with a cognitive load theory of multimodal interaction. The long-term goal of this research is the development of an empirical foundation for proactively guiding flexible and adaptive multimodal system design.