A vision-based sign language recognition system using tied-mixture density HMM

  • Authors:

  • Liang-Guo Zhang;Yiqiang Chen;Gaolin Fang;Xilin Chen;Wen Gao

  • Affiliations:

  • Chinese Academy of Sciences, Beijing, China;Chinese Academy of Sciences, Beijing, China;Harbin Institute of Technology, Harbin, China;Harbin Institute of Technology, Harbin, China;Chinese Academy of Sciences, Beijing, China and Harbin Institute of Technology, Harbin, China

  • Venue:
  • Proceedings of the 6th international conference on Multimodal interfaces
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper, a vision-based medium vocabulary Chinese sign language recognition (SLR) system is presented. The proposed recognition system consists of two modules. In the first module, techniques of robust hands detection, background subtraction and pupils detection are efficiently combined to precisely extract the feature information with the aid of simple colored gloves in the unconstrained environment. Meanwhile, an effective and efficient hierarchical feature description scheme with different scale features to characterize sign language is proposed, where principal component analysis (PCA) is employed to characterize the finger features more elaborately. In the second part, a Tied-Mixture Density Hidden Markov Models (TMDHMM) framework for SLR is proposed, which can speed up the recognition without the significant loss of recognition accuracy compared with the continuous hidden Markov models (CHMM). Experimental results based on 439 frequently used Chinese sign language (CSL) words show that the proposed methods can work well for the medium vocabulary SLR in the environment without special constraints and the recognition accuracy is up to 92.5%.