Robust Real-Time Face Pose and Facial Expression Recovery

Authors:
Zhiwei Zhu;Qiang Ji
Affiliations:
Rensselaer Polytechnic Institute, Troy, NY;Rensselaer Polytechnic Institute, Troy, NY
Venue:
CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1
Year:
2006

Citing 0
Cited 8

Facial expression recognition system based on rigid and non-rigid motion separation and 3D pose estimation

Pattern Recognition
Pose-Invariant Facial Expression Recognition Using Variable-Intensity Templates

International Journal of Computer Vision
Person-independent monocular tracking of face and facial actions with multilinear models

AMFG'07 Proceedings of the 3rd international conference on Analysis and modeling of faces and gestures
Pose-invariant facial expression recognition using variable-intensity templates

ACCV'07 Proceedings of the 8th Asian conference on Computer vision - Volume Part I
Analyzing facial expression by fusing manifolds

ACCV'07 Proceedings of the 8th Asian conference on Computer vision - Volume Part II
Image ratio features for facial expression recognition application

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics - Special issue on game theory
Coupled Gaussian process regression for pose-invariant facial expression recognition

ECCV'10 Proceedings of the 11th European conference on Computer vision: Part II
Smooth adaptive fitting of 3D face model for the estimation of rigid and nonrigid facial motion in video sequences

Image Communication

Quantified Score

Hi-index	0.00

Visualization

Abstract

Face motion is the sum of rigid motion related with face pose and non-rigid motion related with facial expression. Both motions are coupled in the captured image so that they can not be easily recovered from the image directly. In this paper, a novel technique is proposed to recover 3D face pose and facial expression simultaneously from a monocular video sequence in real time. First, twenty-eight salient facial features are detected and tracked robustly under various face orientations and facial expressions. Second, after modelling the coupling between face pose and facial expression in the 2D image as a nonlinear function, a normalized SVD (N-SVD) decomposition technique is proposed to recover the pose and expression parameters analytically. A nonlinear technique is subsequently utilized to refine the solution obtained from the N-SVD technique by imposing the orthonormality constraint on the pose parameters. Compared to the original SVD technique proposed in [1], which is very sensitive to the image noise and numerically unstable in practice, the proposed method can recover the face pose and facial expression robustly and accurately. Finally, the performance of the proposed technique is evaluated in the experiments using both synthetic and real image sequences.