3d gesture recognition applying long short-term memory and contextual knowledge in a CAVE

Authors:
Dejan Arsićc;Luis Roalter;Martin Wöllmer;Florian Eyben;Björn Schuller;Moritz Kaiser;Matthias Kranz;Gerhard Rigoll
Affiliations:
Technische Universität München, Munich, Germany;Technische Universität München, Munich, Germany;Technische Universität München, Munich, Germany;Technische Universität München, Munich, Germany;Technische Universität München, Munich, Germany;Technische Universität München, Munich, Germany;Technische Universität München, Munich, Germany;Technische Universität München, Munich, Germany
Venue:
Proceedings of the 1st ACM international workshop on Multimodal pervasive video analysis
Year:
2010

Citing 6
Cited 0

Learning precise timing with lstm recurrent networks

The Journal of Machine Learning Research
2005 Special Issue: Framewise phoneme classification with bidirectional LSTM and other neural network architectures

Neural Networks - 2005 Special issue: IJCNN 2005
Visual Hand Tracking Algorithms

GMAI '06 Proceedings of the conference on Geometric Modeling and Imaging: New Trends
Long Short-Term Memory

Neural Computation
GMs in On-Line Handwritten Whiteboard Note Recognition: The Influence of Implementation and Modeling

ICDAR '09 Proceedings of the 2009 10th International Conference on Document Analysis and Recognition
Resolving partial occlusions in crowded environments utilizing range data and video cameras

DSP'09 Proceedings of the 16th international conference on Digital Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Virtual reality applications are emerging into various regions of research and entertainment. Although visual and acoustic capabilities are already quite impressive, a wide range of users still criticizes the user interface. Frequently complex and very sensitive input devices are being used, although simple gestures would be preferred. While gesture recognition systems are quite common, see Nintendo's Wii mote, a CAVE has further challenges, as the person can be located in any random position and the gestures are not being performed related to a common fixpoint. Applying an infrared tracking system it is possible to reliably locate the hand and compute 3D trajectories. These are then further analyzed with a Long Short-Term Memory approach, which is able to model sequences of variable length with a higher reliability than HMMs.