Least-Squares Fitting of Two 3-D Point Sets
IEEE Transactions on Pattern Analysis and Machine Intelligence
Performance-driven facial animation
SIGGRAPH '90 Proceedings of the 17th annual conference on Computer graphics and interactive techniques
Proceedings of the 25th annual conference on Computer graphics and interactive techniques
IEEE Transactions on Pattern Analysis and Machine Intelligence
3-D Motion Estimation in Model-Based Facial Image Coding
IEEE Transactions on Pattern Analysis and Machine Intelligence
Modeling, Tracking and Interactive Animation of Faces and Heads Using Input from Video
CA '96 Proceedings of the Computer Animation
CVPR '96 Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96)
ICCV '95 Proceedings of the Fifth International Conference on Computer Vision
Vision-based control of 3D facial animation
Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation
Universal capture: image-based facial animation for "The Matrix Reloaded"
ACM SIGGRAPH 2003 Sketches & Applications
SoftPOSIT: Simultaneous Pose and Correspondence Determination
International Journal of Computer Vision
Spacetime faces: high resolution capture for modeling and animation
ACM SIGGRAPH 2004 Papers
Face transfer with multilinear models
ACM SIGGRAPH 2005 Papers
Efficiently combining positions and normals for precise 3D geometry
ACM SIGGRAPH 2005 Papers
Face Description with Local Binary Patterns: Application to Face Recognition
IEEE Transactions on Pattern Analysis and Machine Intelligence
Multi-scale capture of facial geometry and motion
ACM SIGGRAPH 2007 papers
Face/Off: live facial puppetry
Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation
The Digital Emily project: photoreal facial modeling and animation
ACM SIGGRAPH 2009 Courses
Temporal upsampling of performance geometry using photometric alignment
ACM Transactions on Graphics (TOG)
ACM SIGGRAPH 2010 papers
High resolution passive facial performance capture
ACM SIGGRAPH 2010 papers
ECCV'10 Proceedings of the 11th European conference on Computer vision: Part I
Real-time combined 2D+3D active appearance models
CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
Deformable Model Fitting by Regularized Landmark Mean-Shift
International Journal of Computer Vision
Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition
ACM SIGGRAPH 2011 papers
High-quality passive facial performance capture using anchor frames
ACM SIGGRAPH 2011 papers
Realtime performance-based facial animation
ACM SIGGRAPH 2011 papers
Proceedings of the 2011 SIGGRAPH Asia Conference
Dense versus Sparse Approaches for Estimating the Fundamental Matrix
International Journal of Computer Vision
Real-time facial feature detection using conditional regression forests
CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
A data-driven approach for facial expression synthesis in video
CVPR '12 Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Modeling temporal coherence for optical flow
ICCV '11 Proceedings of the 2011 International Conference on Computer Vision
Lightweight binocular facial performance capture under uncontrolled lighting
ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH Asia 2012
Online modeling for realtime facial animation
ACM Transactions on Graphics (TOG) - SIGGRAPH 2013 Conference Proceedings
3D shape regression for real-time facial animation
ACM Transactions on Graphics (TOG) - SIGGRAPH 2013 Conference Proceedings
Realtime facial animation with on-the-fly correctives
ACM Transactions on Graphics (TOG) - SIGGRAPH 2013 Conference Proceedings
Hi-index | 0.00 |
Detailed facial performance geometry can be reconstructed using dense camera and light setups in controlled studios. However, a wide range of important applications cannot employ these approaches, including all movie productions shot from a single principal camera. For post-production, these require dynamic monocular face capture for appearance modification. We present a new method for capturing face geometry from monocular video. Our approach captures detailed, dynamic, spatio-temporally coherent 3D face geometry without the need for markers. It works under uncontrolled lighting, and it successfully reconstructs expressive motion including high-frequency face detail such as folds and laugh lines. After simple manual initialization, the capturing process is fully automatic, which makes it versatile, lightweight and easy-to-deploy. Our approach tracks accurate sparse 2D features between automatically selected key frames to animate a parametric blend shape model, which is further refined in pose, expression and shape by temporally coherent optical flow and photometric stereo. We demonstrate performance capture results for long and complex face sequences captured indoors and outdoors, and we exemplify the relevance of our approach as an enabling technology for model-based face editing in movies and video, such as adding new facial textures, as well as a step towards enabling everyone to do facial performance capture with a single affordable camera.