QuickSet: multimodal interaction for simulation set-up and control

Authors:
Philip R. Cohen;Michael Johnston;David McGee;Sharon Oviatt;Jay Pittman;Ira Smith;Liang Chen;Josh Clow
Affiliations:
Oregon Graduate Institute of Science and Technology, Portland, OR;Oregon Graduate Institute of Science and Technology, Portland, OR;Oregon Graduate Institute of Science and Technology, Portland, OR;Oregon Graduate Institute of Science and Technology, Portland, OR;Oregon Graduate Institute of Science and Technology, Portland, OR;Oregon Graduate Institute of Science and Technology, Portland, OR;Oregon Graduate Institute of Science and Technology, Portland, OR;Oregon Graduate Institute of Science and Technology, Portland, OR
Venue:
ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Year:
1997

Citing 14
Cited 11

Synergistic use of direct manipulation and natural language

CHI '89 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Intelligent multi-media interface technology

Intelligent user interfaces
The logic of typed feature structures

The logic of typed feature structures
The role of natural language in a multimodal interface

UIST '92 Proceedings of the 5th annual ACM symposium on User interface software and technology
NPSNET: constructing a 3D virtual world

I3D '92 Proceedings of the 1992 symposium on Interactive 3D graphics
Integrating simultaneous input from speech, gaze, and hand gestures

Intelligent multimedia interfaces
Surround-screen projection-based virtual reality: the design and implementation of the CAVE

SIGGRAPH '93 Proceedings of the 20th annual conference on Computer graphics and interactive techniques
Toward interface design for human language technology: modality and structure as determinants of linguistic complexity

ISSD-93 Selected papers presented at the international symposium on Spoken dialogue
Multimodal interfaces for dynamic interactive maps

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Integration and synchronization of input modes during multimodal human-computer interaction

Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Integrated interfaces for decision-support with simulation

WSC '91 Proceedings of the 23rd conference on Winter simulation
Multimodal Maps: An Agent-Based Approach

Multimodal Human-Computer Communication, Systems, Techniques, and Experiments
Unification-based multimodal integration

ACL '98 Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics
Multimodal interactive maps: designing for human performance

Human-Computer Interaction

The Neem Platform: An Extensible Framework for the Development of Perceptual Collaborative Applications

EDCIS '02 Proceedings of the First International Conference on Engineering and Deployment of Cooperative Information Systems
Perceptual Collaboration in Neem

ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
The Neem Platform: An Evolvable Framework for Perceptual Collaborative Applications

Journal of Intelligent Information Systems
Multimodal information access across multiple devices

Mobility '07 Proceedings of the 4th international conference on mobile technology, applications, and systems and the 1st international symposium on Computer human interaction in mobile technology
Usability framework for the design and evaluation of multimodal interaction

BCS-HCI '08 Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction - Volume 2
Design of communication in multimodal web interfaces

Proceedings of the 27th ACM international conference on Design of communication
Multimodal integration: a biological view

IJCAI'01 Proceedings of the 17th international joint conference on Artificial intelligence - Volume 2
Onomatopen: painting using onomatopoeia

ICEC'10 Proceedings of the 9th international conference on Entertainment computing
Fusion in multimodal interactive systems: an HMM-based algorithm for user-induced adaptation

Proceedings of the 4th ACM SIGCHI symposium on Engineering interactive computing systems
An exploration of gesture-speech multimodal patterns for touch interfaces

Proceedings of the 3rd International Conference on Human Computer Interaction
GlueTK: a framework for multi-modal, multi-display human-machine-interaction

Proceedings of the 2013 international conference on Intelligent user interfaces

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a novel multimodal system applied to the setup and control of distributed interactive simulations. We have developed the QuickSet prototype, a pen/voice system running on a hand-held PC, communicating through a distributed agent architecture to NRaD's1 LeatherNet system, a distributed interactive training simulator built for the US Marine Corps (USMC). The paper briefly describes the system and illustrates its use in multimodal simulation setup.