Facial expression recognition system based on rigid and non-rigid motion separation and 3D pose estimation

  • Authors:
  • Te-Hsun Wang;Jenn-Jier James Lien

  • Affiliations:
  • Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan 70101, Taiwan, ROC;Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan 70101, Taiwan, ROC

  • Venue:
  • Pattern Recognition
  • Year:
  • 2009

Quantified Score

Hi-index 0.01

Visualization

Abstract

This study presents a facial expression recognition system which separates the non-rigid facial expression from the rigid head rotation and estimates the 3D rigid head rotation angle in real time. The extracted trajectories of the feature points contain both rigid head motion components and non-rigid facial expression motion components. A 3D virtual face model is used to obtain accurate estimation of the head rotation angle such that the non-rigid motion components can be precisely separated to enhance the facial expression recognition performance. The separation performance of the proposed system is further improved through the use of a restoration mechanism designed to recover feature points lost during large pan rotations. Having separated the rigid and non-rigid motions, hidden Markov models (HMMs) are employed to recognize a prescribed set of facial expressions defined in terms of facial action coding system (FACS) action units (AUs).