Interaction techniques using hand tracking and speech recognition
Multimedia interface design
Gestures with speech for graphic manipulation
International Journal of Man-Machine Studies
Integrating simultaneous input from speech, gaze, and hand gestures
Intelligent multimedia interfaces
Research in multimedia and multimodal parsing and generation
Artificial Intelligence Review - Special issue on integration of natural language and vision processing: intelligent multimedia
Detection of Fingertips in Human Hand Movement Sequences
Proceedings of the International Gesture Workshop on Gesture and Sign Language in Human-Computer Interaction
“Put-that-there”: Voice and gesture at the graphics interface
SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques
Avocado: A Distributed Virtual Reality Framework
VR '99 Proceedings of the IEEE Virtual Reality
Designing Transition Networks for Multimodal VR-Interactions Using a Markup Language
ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
A user interface framework for multimodal VR interactions
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
Explicit task representation based on gesture interaction
MMUI '05 Proceedings of the 2005 NICTA-HCSNet Multimodal User Interaction Workshop - Volume 57
Knowledge in the loop: semantics representation for multimodal simulative environments
SG'05 Proceedings of the 5th international conference on Smart Graphics
Hi-index | 0.00 |
This article presents a gesture detection and analysis framework for modelling multimodal interactions. It is particulary designed for its use in Virtual Reality (VR) applications and contains an abstraction layer for different sensor hardware. Using the framework, gestures are described by their characteristic spatio-temporal features which are on the lowest level calculated by simple predefined detector modules or nodes. These nodes can be connected by a data routing mechanism to perform more elaborate evaluation functions, therewith establishing complex detector nets. Typical problems that arise from the time-dependent invalidation of multimodal utterances under immersive conditions lead to the development of pre-evaluation concepts that as well support their integration into scene graph based systems to support traversal-type access. Examples of realized interactions illustrate applications which make use of the described concepts.