Face replacement with large-pose differences

Authors:
Yuan Lin;Qian Lin;Feng Tang;Shengjin Wang
Affiliations:
Tsinghua University, Beijing, China;USA HP Labs, Multimedia Interaction and Understanding Lab, Palo Alto, CA, USA;USA HP Labs, Multimedia Interaction and Understanding Lab, Palo Alto, CA, USA;Tsinghua University, Beijing, China
Venue:
Proceedings of the 20th ACM international conference on Multimedia
Year:
2012

Citing 3
Cited 0

A multiresolution spline with application to image mosaics

ACM Transactions on Graphics (TOG)
Color Transfer between Images

IEEE Computer Graphics and Applications
Face swapping: automatically replacing faces in photographs

ACM SIGGRAPH 2008 papers

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we present a novel face replacement system exchanging faces with large-pose differences. Traditional 2D image based face replacement can only replace faces with similar pose and appearance. This significantly limits the application of face replacement. In this paper, we propose to build a 3D head model from a single frontal face photo. The automatically constructed 3D head can be rendered under arbitrary poses and illuminations. This makes it possible to do swapping for faces with large pose variations. In the demo, the user captures a frontal face image using a capture device such as a webcam or a smartphone, and then the algorithm can automatically build the 3D model using feature detection, face alignment and reconstruction. This 3D model is used to swap to any other target face photo the user selects. While our system is automatic, we also provide interactive tools for the user to adjust the feature detection to enhance the results.