Mulitmodal interaction for distributed interactive simulation

Authors:
Philip R. Cohen;Michael Johnston;David McGee;Sharon Oviatt;Jay Pittman;Ira Smith;Liang Chen;Josh Clow
Affiliations:
Center for Human Computer Communication, Oregon Graduate Institute of Science and Technology, Portland, OR;Center for Human Computer Communication, Oregon Graduate Institute of Science and Technology, Portland, OR;Center for Human Computer Communication, Oregon Graduate Institute of Science and Technology, Portland, OR;Center for Human Computer Communication, Oregon Graduate Institute of Science and Technology, Portland, OR;Center for Human Computer Communication, Oregon Graduate Institute of Science and Technology, Portland, OR;Center for Human Computer Communication, Oregon Graduate Institute of Science and Technology, Portland, OR;Center for Human Computer Communication, Oregon Graduate Institute of Science and Technology, Portland, OR;Center for Human Computer Communication, Oregon Graduate Institute of Science and Technology, Portland, OR
Venue:
AAAI'97/IAAI'97 Proceedings of the fourteenth national conference on artificial intelligence and ninth conference on Innovative applications of artificial intelligence
Year:
1997

Citing 18
Cited 2

Recognizing handwritten text

CHI '91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Intelligent multi-media interface technology

Intelligent user interfaces
The logic of typed feature structures

The logic of typed feature structures
The role of natural language in a multimodal interface

UIST '92 Proceedings of the 5th annual ACM symposium on User interface software and technology
NPSNET: constructing a 3D virtual world

I3D '92 Proceedings of the 1992 symposium on Interactive 3D graphics
Integrating simultaneous input from speech, gaze, and hand gestures

Intelligent multimedia interfaces
Surround-screen projection-based virtual reality: the design and implementation of the CAVE

SIGGRAPH '93 Proceedings of the 20th annual conference on Computer graphics and interactive techniques
Software agents

Communications of the ACM
KQML as an agent communication language

CIKM '94 Proceedings of the third international conference on Information and knowledge management
Cooperating heterogeneous systems: a blackboard-based meta approach

Cooperating heterogeneous systems: a blackboard-based meta approach
Multimodal interfaces for dynamic interactive maps

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Integration and synchronization of input modes during multimodal human-computer interaction

Proceedings of the ACM SIGCHI Conference on Human factors in computing systems
Multimodal Maps: An Agent-Based Approach

Multimodal Human-Computer Communication, Systems, Techniques, and Experiments
“Put-that-there”: Voice and gesture at the graphics interface

SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
CommandTalk: a spoken-language interface for battlefield simulations

ANLC '97 Proceedings of the fifth conference on Applied natural language processing
Building an application framework for speech and pen input integration in multimodal learning interfaces

ICASSP '96 Proceedings of the Acoustics, Speech, and Signal Processing, 1996. on Conference Proceedings., 1996 IEEE International Conference - Volume 06
Multimodal interactive maps: designing for human performance

Human-Computer Interaction
Toward a semantics for an agent communications language based on speech0-acts

AAAI'96 Proceedings of the thirteenth national conference on Artificial intelligence - Volume 1

Confirmation in multimodal systems

COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 2
An agent-based framework for sketched symbol interpretation

Journal of Visual Languages and Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents an emerging application of Artificial Intelligence research to distributed interactive simulations, with the goal of reducing exercise generation time and effort, yet maximizing training effectiveness. We have developed the QuickSet prototype, a pen/voice system running on a hand-held PC, communicating via wireless LAN through an agent architecture to NRaD's LeatherNet system, a distributed interactive training simulator built for the US Marine Corps. The paper describes our novel multi modal integration strategy offering mutual compensation among modalities, as well as QuickSet's agent-based infrastructure, and provides an example of multimodal simulation setup. Finally, we discuss our applications experience and lessons learned.