Estimating 3D Face Model and Facial Deformation from a Single Image Based on Expression Manifold Optimization

  • Authors:
  • Shu-Fan Wang;Shang-Hong Lai

  • Affiliations:
  • Department of Computer Science, National Tsing Hua University, Taiwan;Department of Computer Science, National Tsing Hua University, Taiwan

  • Venue:
  • ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Facial expression modeling is central to facial expression recognition and expression synthesis for facial animation. Previous works reported that modeling the facial expression with low-dimensional manifold is more appropriate than using a linear subspace. In this paper, we propose a manifold-based 3D face reconstruction approach to estimating the 3D face model and the associated expression deformation from a single face image. In the training phase, we build a nonlinear 3D expression manifold from a large set of 3D facial expression models to represent the facial shape deformations due to facial expressions. Then a Gaussian mixture model in this manifold is learned to represent the distribution of expression deformation. By combining the merits of morphable neutral face model and the low-dimensional expression manifold, we propose a new algorithm to reconstruct the 3D face geometry as well as the 3D shape deformation from a single face image with expression in an energy minimization framework. Experimental results on CMU-PIE image database and FG-Net video database are shown to validate the effectiveness and accuracy of the proposed algorithm.