Speech and 2d deictic gesture reference to virtual scenes

Authors:
Niels Ole Bernsen
Affiliations:
Natural Interactive Systems Laboratory, Odense, Denmark
Venue:
PIT'06 Proceedings of the 2006 international tutorial and research conference on Perception and Interactive Technologies
Year:
2006

Citing 3
Cited 0

“Put-that-there”: Voice and gesture at the graphics interface

SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
Visual perception, language and gesture: a model for their understanding in multimodal dialogue systems

Signal Processing - Special section: Multimodal human-computer interfaces
Fusion of children's speech and 2D gestures when conversing with 3D characters

Signal Processing - Special section: Multimodal human-computer interfaces

Quantified Score

Hi-index	0.00

Visualization

Abstract

Humans make ample use of deictic gesture and spoken reference in referring to perceived phenomena in the spatial environment, such as visible objects, sound sources, tactile objects, or even sources of smell and taste. Multimodal and natural interactive systems developers are beginning to face the challenges involved in making systems correctly interpret user input belonging to this general class of multimodal references. This paper addresses a first fragment of the general problem, i.e., spoken and/or 2D on-screen deictic gesture reference to graphics output scenes. The approach is to confront existing sketchy theory with new data and generalise the results to what may be a more comprehensive understanding of the problem.