Performance-driven facial animation
SIGGRAPH '90 Proceedings of the 17th annual conference on Computer graphics and interactive techniques
A Method for Registration of 3-D Shapes
IEEE Transactions on Pattern Analysis and Machine Intelligence - Special issue on interpretation of 3-D scenes—part II
Model-based object pose in 25 lines of code
International Journal of Computer Vision - Special issue: image understanding research at the University of Maryland
A limited memory algorithm for bound constrained optimization
SIAM Journal on Scientific Computing
Synthesizing realistic facial expressions from photographs
Proceedings of the 25th annual conference on Computer graphics and interactive techniques
A morphable model for the synthesis of 3D faces
Proceedings of the 26th annual conference on Computer graphics and interactive techniques
Optical Flow Constraints on Deformable Models with Applications to Face Tracking
International Journal of Computer Vision
A Flexible New Technique for Camera Calibration
IEEE Transactions on Pattern Analysis and Machine Intelligence
Random projection in dimensionality reduction: applications to image and text data
Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining
Modeling, Tracking and Interactive Animation of Faces and Heads Using Input from Video
CA '96 Proceedings of the Computer Animation
Vision-based control of 3D facial animation
Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation
Spacetime faces: high resolution capture for modeling and animation
ACM SIGGRAPH 2004 Papers
Face transfer with multilinear models
ACM SIGGRAPH 2005 Papers
A Closed-Form Solution to Non-Rigid Shape and Motion Recovery
International Journal of Computer Vision
2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting
International Journal of Computer Vision
Face/Off: live facial puppetry
Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation
ACM SIGGRAPH 2010 papers
High-quality single-shot capture of facial geometry
ACM SIGGRAPH 2010 papers
High resolution passive facial performance capture
ACM SIGGRAPH 2010 papers
Direct Manipulation Blendshapes
IEEE Computer Graphics and Applications
Expression flow for 3D-aware face component transfer
ACM SIGGRAPH 2011 papers
Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition
ACM SIGGRAPH 2011 papers
High-quality passive facial performance capture using anchor frames
ACM SIGGRAPH 2011 papers
Realtime performance-based facial animation
ACM SIGGRAPH 2011 papers
Content retargeting using parameter-parallel facial layers
SCA '11 Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation
Compression and direct manipulation of complex blendshape models
Proceedings of the 2011 SIGGRAPH Asia Conference
A Coupled Statistical Model for Face Shape Recovery From Brightness Images
IEEE Transactions on Image Processing
Face alignment by Explicit Shape Regression
CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Facial action transfer with personalized bilinear regression
ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part II
Robust and accurate shape model fitting using random forest regression voting
ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part VII
Reconstructing detailed dynamic face geometry from monocular video
ACM Transactions on Graphics (TOG)
| Hi-index | 0.00 |
We present a real-time performance-driven facial animation system based on 3D shape regression. In this system, the 3D positions of facial landmark points are inferred by a regressor from 2D video frames of an ordinary web camera. From these 3D points, the pose and expressions of the face are recovered by fitting a user-specific blendshape model to them. The main technical contribution of this work is the 3D regression algorithm that learns an accurate, user-specific face alignment model from an easily acquired set of training data, generated from images of the user performing a sequence of predefined facial poses and expressions. Experiments show that our system can accurately recover 3D face shapes even for fast motions, non-frontal faces, and exaggerated expressions. In addition, some capacity to handle partial occlusions and changing lighting conditions is demonstrated.