Robust sign language recognition by combining manual and non-manual features based on conditional random field and support vector machine

Authors:
Hee-Deok Yang;Seong-Whan Lee
Affiliations:
-;-
Venue:
Pattern Recognition Letters
Year:
2013

Citing 9
Cited 0

A Parallel Multistream Model for Integration of Sign Language Recognition and Lip Motion

ICMI '00 Proceedings of the Third International Conference on Advances in Multimodal Interfaces
Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning

IEEE Transactions on Pattern Analysis and Machine Intelligence
A belief-based sequential fusion approach for fusing manual signs and non-manual signals

Pattern Recognition
Sign Language Spotting with a Threshold Model Based on Conditional Random Fields

IEEE Transactions on Pattern Analysis and Machine Intelligence
Simultaneous spotting of signs and fingerspellings based on hierarchical conditional random fields and boostmap embeddings

Pattern Recognition
A Robust Method for Hand Gesture Segmentation and Recognition Using Forward Spotting Scheme in Conditional Random Fields

ICPR '10 Proceedings of the 2010 20th International Conference on Pattern Recognition
American sign language recognition with the kinect

ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Kernel-based representation for 2D/3D motion trajectory retrieval and classification

Pattern Recognition
Sign language recognition using sub-units

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.10

Visualization

Abstract

The sign language is composed of two categories of signals: manual signals such as signs and fingerspellings and non-manual ones such as body gestures and facial expressions. This paper proposes a new method for recognizing manual signals and facial expressions as non-manual signals. The proposed method involves the following three steps: First, a hierarchical conditional random field is used to detect candidate segments of manual signals. Second, the BoostMap embedding method is used to verify hand shapes of segmented signs and to recognize fingerspellings. Finally, the support vector machine is used to recognize facial expressions as non-manual signals. This final step is taken when there is some ambiguity in the previous two steps. The experimental results indicate that the proposed method can accurately recognize the sign language at an 84% rate based on utterance data.