Performance-driven facial animation
SIGGRAPH '90 Proceedings of the 17th annual conference on Computer graphics and interactive techniques
Proceedings of the 25th annual conference on Computer graphics and interactive techniques
Multiple view geometry in computer visiond
Multiple view geometry in computer visiond
International Journal of Computer Vision
CVPR '96 Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96)
Recovering 3-D shape and reflectance from a small number of photographs
EGRW '03 Proceedings of the 14th Eurographics workshop on Rendering
Universal capture: image-based facial animation for "The Matrix Reloaded"
ACM SIGGRAPH 2003 Sketches & Applications
Efficient Graph-Based Image Segmentation
International Journal of Computer Vision
Spacetime faces: high resolution capture for modeling and animation
ACM SIGGRAPH 2004 Papers
Efficiently combining positions and normals for precise 3D geometry
ACM SIGGRAPH 2005 Papers
Photometric Stereo with General, Unknown Lighting
International Journal of Computer Vision
Multi-scale capture of facial geometry and motion
ACM SIGGRAPH 2007 papers
Poisson surface reconstruction
SGP '06 Proceedings of the fourth Eurographics symposium on Geometry processing
3-D Reconstruction of Shaded Objects from Multiple Images Under Unknown Illumination
International Journal of Computer Vision
ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part III
Efficient Dense Scene Flow from Sparse or Dense Stereo Data
ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data
ACM Transactions on Graphics (TOG)
The Digital Emily project: photoreal facial modeling and animation
ACM SIGGRAPH 2009 Courses
Temporal upsampling of performance geometry using photometric alignment
ACM Transactions on Graphics (TOG)
High-quality single-shot capture of facial geometry
ACM SIGGRAPH 2010 papers
High resolution passive facial performance capture
ACM SIGGRAPH 2010 papers
Joint estimation of motion, structure and geometry from stereo sequences
ECCV'10 Proceedings of the 11th European conference on Computer vision: Part IV
Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition
ACM SIGGRAPH 2011 papers
High-quality passive facial performance capture using anchor frames
ACM SIGGRAPH 2011 papers
Realtime performance-based facial animation
ACM SIGGRAPH 2011 papers
International Journal of Computer Vision
Dense versus Sparse Approaches for Estimating the Fundamental Matrix
International Journal of Computer Vision
Self-calibrated, Multi-spectral Photometric Stereo for 3D Face Capture
International Journal of Computer Vision
High-quality shape from multi-view stereo and shading under general illumination
CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
Basis constrained 3D scene flow on a dynamic proxy
ICCV '11 Proceedings of the 2011 International Conference on Computer Vision
Shading-based dynamic shape refinement from multi-view video under general illumination
ICCV '11 Proceedings of the 2011 International Conference on Computer Vision
Online modeling for realtime facial animation
ACM Transactions on Graphics (TOG) - SIGGRAPH 2013 Conference Proceedings
Realtime facial animation with on-the-fly correctives
ACM Transactions on Graphics (TOG) - SIGGRAPH 2013 Conference Proceedings
Reconstructing detailed dynamic face geometry from monocular video
ACM Transactions on Graphics (TOG)
Sparse localized deformation components
ACM Transactions on Graphics (TOG)
On-set performance capture of multiple actors with a stereo camera
ACM Transactions on Graphics (TOG)
Hi-index | 0.00 |
Recent progress in passive facial performance capture has shown impressively detailed results on highly articulated motion. However, most methods rely on complex multi-camera set-ups, controlled lighting or fiducial markers. This prevents them from being used in general environments, outdoor scenes, during live action on a film set, or by freelance animators and everyday users who want to capture their digital selves. In this paper, we therefore propose a lightweight passive facial performance capture approach that is able to reconstruct high-quality dynamic facial geometry from only a single pair of stereo cameras. Our method succeeds under uncontrolled and time-varying lighting, and also in outdoor scenes. Our approach builds upon and extends recent image-based scene flow computation, lighting estimation and shading-based refinement algorithms. It integrates them into a pipeline that is specifically tailored towards facial performance reconstruction from challenging binocular footage under uncontrolled lighting. In an experimental evaluation, the strong capabilities of our method become explicit: We achieve detailed and spatio-temporally coherent results for expressive facial motion in both indoor and outdoor scenes -- even from low quality input images recorded with a hand-held consumer stereo camera. We believe that our approach is the first to capture facial performances of such high quality from a single stereo rig and we demonstrate that it brings facial performance capture out of the studio, into the wild, and within the reach of everybody.