A new probabilistic model for recognizing signs with systematic modulations

Authors:
Sylvie C. W. Ong;Surendra Ranganath
Affiliations:
Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore;Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore
Venue:
AMFG'07 Proceedings of the 3rd international conference on Analysis and modeling of faces and gestures
Year:
2007

Citing 10
Cited 0

Motion analysis of grammatical processes in a visual-gestural language

Proc. of the ACM SIGGRAPH/SIGART interdisciplinary workshop on Motion: representation and perception
The Hierarchical Hidden Markov Model: Analysis and Applications

Machine Learning
A Method for Analyzing Spatial Relationships Between Words in Sign Language Recognition

GW '99 Proceedings of the International Gesture Workshop on Gesture-Based Communication in Human-Computer Interaction
ARGo: An Architecture for Sign Language Recognition and Interpretation

Proceedings of Gesture Workshop on Progress in Gestural Interaction
Coupled hidden Markov models for complex action recognition

CVPR '97 Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97)
Layered Representations for Human Activity Recognition

ICMI '02 Proceedings of the 4th IEEE International Conference on Multimodal Interfaces
Dynamic bayesian networks: representation, inference and learning

Dynamic bayesian networks: representation, inference and learning
American sign language recognition: reducing the complexity of the task with phoneme-based modeling and parallel hidden markov models

American sign language recognition: reducing the complexity of the task with phoneme-based modeling and parallel hidden markov models
Understanding gestures with systematic variations in movement dynamics

Pattern Recognition
Asynchrony modeling for audio-visual speech recognition

HLT '02 Proceedings of the second international conference on Human Language Technology Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper addresses an aspect of sign language (SL) recognition that has largely been overlooked in previous work and yet is integral to signed communication. It is the most comprehensive work to-date on recognizing complex variations in sign appearances due to grammatical processes (inflections) which systematically modulate the temporal and spatial dimensions of a root sign word to convey information in addition to lexical meaning. We propose a novel dynamic Bayesian network - the Multichannel Hierarchical Hidden Markov Model (MH-HMM)- as a modelling and recognition framework for continuously signed sentences that include modulated signs. This models the hierarchical, sequential and parallel organization in signing while requiring synchronization between parallel data streams at sign boundaries. Experimental results using particle filtering for decoding demonstrate the feasibility of using the MH-HMM for recognizing inflected signs in continuous sentences.