The efficiency of multimodal interaction for a map-based task

Authors:
Philip Cohen;David McGee;Josh Clow
Affiliations:
Center for Human-Computer Communication, Oregon Graduate Institute of Science & Technology, Beaverton, Oregon;Center for Human-Computer Communication, Oregon Graduate Institute of Science & Technology, Beaverton, Oregon;Center for Human-Computer Communication, Oregon Graduate Institute of Science & Technology, Beaverton, Oregon
Venue:
ANLC '00 Proceedings of the sixth conference on Applied natural language processing
Year:
2000

Citing 12
Cited 19

Synergistic use of direct manipulation and natural language

CHI '89 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
The utility of speech input in user-computer interfaces

International Journal of Man-Machine Studies
QuickSet: multimodal interaction for distributed applications

MULTIMEDIA '97 Proceedings of the fifth ACM international conference on Multimedia
Patterns of entry and correction in large vocabulary continuous speech recognition systems

Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Mutual disambiguation of recognition errors in a multimodel architecture

Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Model-based and empirical evaluation of multimodal interactive error correction

Proceedings of the SIGCHI conference on Human Factors in Computing Systems
Multimodal Interaction for 2D and 3D Environments

IEEE Computer Graphics and Applications
CommandTalk: a spoken-language interface for battlefield simulations

ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Unification-based multimodal integration

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Natural vs. precise concise languages for human operation of computers: research issues and experimental approaches

ACL '80 Proceedings of the 18th annual meeting on Association for Computational Linguistics
Mode preference in a simple data-retrieval task

HLT '93 Proceedings of the workshop on Human Language Technology
Multimodal interactive maps: designing for human performance

Human-Computer Interaction

Comparing paper and tangible, multimodal tools

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Partial Information in Multimodal Dialogue

ICMI '00 Proceedings of the Third International Conference on Advances in Multimodal Interfaces
Multimodal interfaces

The human-computer interaction handbook
Tangible multimodal interfaces for safety-critical applications

Communications of the ACM - Multimodal interfaces that flex, adapt, and persist
Human-centered collaborative interaction

Proceedings of the 1st ACM international workshop on Human-centered multimedia
Personalised maps in multimodal mobile GIS

International Journal of Web Engineering and Technology
HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces

HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces
Explorative studies on multimodal interaction in a PDA- and desktop-based scenario

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
A three-dimensional characterization space of software components for rapidly developing multimodal interfaces

ICMI '08 Proceedings of the 10th international conference on Multimodal interfaces
Steps in Identifying Interaction Design Patterns for Multimodal Systems

HCSE-TAMODIA '08 Proceedings of the 2nd Conference on Human-Centered Software Engineering and 7th International Workshop on Task Models and Diagrams
Context shifts: extending the meanings of physical objects with language

Human-Computer Interaction
Hands-free, speech-based navigation during dictation: difficulties, consequences, and solutions

Human-Computer Interaction
Exploring Multimodal Navigation Aids for Mobile Users

INTERACT '09 Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part I
Multimodal interaction: a new focal area for AI

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Adding speech recognition support to UML tools

Journal of Visual Languages and Computing
Information display of wearable devices through sound feedback of wearable computing

HCI'07 Proceedings of the 12th international conference on Human-computer interaction: interaction platforms and techniques
Multimodal interaction: A suitable strategy for including older users?

Interacting with Computers
Toward natural interaction in the real world: real-time gesture recognition

International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
A comparison between spoken queries and menu-based interfaces for in-car digital music selection

INTERACT'05 Proceedings of the 2005 IFIP TC13 international conference on Human-Computer Interaction

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper compares the efficiency of using a standard direct-manipulation graphical user interface (GUI) with that of using the QuickSet pen/voice multimodal interface for supporting a military task. In this task, a user places military units and control measures (e.g., various types of lines, obstacles, objectives) on a map. Four military personnel designed and entered their own simulation scenarios via both interfaces. Analyses revealed that the multimodal interface led to an average 3.5-fold speed improvement in the average entity creation time, including all error handling. The mean time to repair errors also was 4.3 times faster when interacting multimodally. Finally, all subjects reported a strong preference for multimodal interaction. These results indicate a substantial efficiency advantage for multimodal over GUI-based interaction during map-based tasks.