Toward Natural Gesture/Speech Control of a Large Display
EHCI '01 Proceedings of the 8th IFIP International Conference on Engineering for Human-Computer Interaction
Prosody Based Co-analysis for Continuous Recognition of Coverbal Gestures
ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
Elvis: situated speech and gesture understanding for a robotic chandelier
Proceedings of the 6th international conference on Multimodal interfaces
Exploiting prosodic structuring of coverbal gesticulation
Proceedings of the 6th international conference on Multimodal interfaces
Multimodal model integration for sentence unit detection
Proceedings of the 6th international conference on Multimodal interfaces
Inferring body pose using speech content
ICMI '05 Proceedings of the 7th international conference on Multimodal interfaces
The recognition and comprehension of hand gestures: a review and research agenda
ZiF'06 Proceedings of the Embodied communication in humans and machines, 2nd ZiF research group international conference on Modeling communication with robots and virtual humans
Toward natural interaction in the real world: real-time gesture recognition
International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
Improving continuous gesture recognition with spoken prosody
CVPR'03 Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition
A hierarchical approach to continuous gesture analysis for natural multi-modal interaction
Proceedings of the 14th ACM international conference on Multimodal interaction
Context-based conversational hand gesture classification in narrative interaction
Proceedings of the 15th ACM on International conference on multimodal interaction
| Hi-index | 0.00 |
In order to incorporate naturalness in the design of Human Computer Interfaces (HCI), it is desirable to develop recognition techniques capable of handling continuous natural gesture and speech inputs. Though many different researchers have reported high recognition rates for gesture recognition using Hidden Markov Models (HMMs), the gestures used are mostly pre-defined and are bound with syntactical and grammatical constraints. But natural gestures do not string together in syntactical bindings. Moreover, strict classification of natural gestures is not feasible.In this paper we have examined hand gestures made in a very natural domain, that of a weather person narrating in front of a weather map. The gestures made by the weather person are embedded in a narration. This provides us with abundant data from an uncontrolled environment to study the interaction between speech and gesture in the context of a display. We hypothesize that this domain is very similar to that of a natural human-computer interface. We present an HMMs architecture for continuous gesture recognition framework and keyword spotting. To explore the relation between gesture and speech, we conducted a statistical co-occurrence analysis of different gestures with a selected set of spoken keywords. We then demonstrate how this co-occurrence analysis can be exploited to improve the performance of continuous gesture recognition.