Exploiting phonological constraints for handshape inference in ASL video

Authors:
A. Thangali;J. P. Nash;S. Sclaroff;C. Neidle
Affiliations:
Comput. Sci. Dept., Boston Univ., Boston, MA, USA;Linguistics Program, Boston Univ., Boston, MA, USA;Comput. Sci. Dept., Boston Univ., Boston, MA, USA;Linguistics Program, Boston Univ., Boston, MA, USA
Venue:
CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
Year:
2011

Citing 0
Cited 2

Genetic eigenhand selection for handshape classification based on compact hand extraction

Engineering Applications of Artificial Intelligence
Dynamic affine-invariant shape-appearance handshape features and classification in sign language videos

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Handshape is a key linguistic component of signs, and thus, handshape recognition is essential to algorithms for sign language recognition and retrieval. In this work, linguistic constraints on the relationship between start and end handshapes are leveraged to improve handshape recognition accuracy. A Bayesian network formulation is proposed for learning and exploiting these constraints, while taking into consideration inter-signer variations in the production of particular handshapes. A Variational Bayes formulation is employed for supervised learning of the model parameters. A non-rigid image alignment algorithm, which yields improved robustness to variability in handshape appearance, is proposed for computing image observation likelihoods in the model. The resulting handshape inference algorithm is evaluated using a dataset of 1500 lexical signs in American Sign Language (ASL), where each lexical sign is produced by three native ASL signers.