Facial phoneme extraction for taiwanese sign language recognition

Authors:
Shi-Hou Lin;Pi-Fuei Hsieh;Chung-Hsien Wu
Affiliations:
Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan;Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan;Department of Computer Science and Information Engineering, National Cheng Kung University, Taiwan
Venue:
ACII'05 Proceedings of the First international conference on Affective Computing and Intelligent Interaction
Year:
2005

Citing 3
Cited 0

Recognizing Action Units for Facial Expression Analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence
Generating realistic facial expressions with wrinkles for model-based coding

Computer Vision and Image Understanding
Active Tracking and Cloning of Facial Expressions Using Spatio-Temporal Information

ICTAI '02 Proceedings of the 14th IEEE International Conference on Tools with Artificial Intelligence

Quantified Score

Hi-index	0.00

Visualization

Abstract

We have developed a system that recognizes the facial expressions in Taiwanese Sign Language (TSL) using a phoneme-based strategy. A facial expression is decomposed into three facial phonemes of eyebrow, eye, and mouth. A fast method is proposed for locating facial phonemes. The shapes of the phonemes were then matched by the deformable template method, giving feature points representing the corresponding phonemes. The trajectories of the feature points were tracked along the video image sequence and combined to recognize the type of facial expression. The tracking techniques and the feature points have been tailored for facial features in TSL. For example, the template matching methods have been simplified for tracking eyebrows and eyes. The mouth was tracked using the optical flow method, taking lips as homogeneous patches. The experiment has been conducted on 70 image sequences covering seven facial expressions. The average recognition rate is 83.3%.