Model-based stereo with occlusions

  • Authors:
  • Fabiano Romeiro;Todd Zickler

  • Affiliations:
  • School of Engineering and Applied Sciences, Harvard University;School of Engineering and Applied Sciences, Harvard University

  • Venue:
  • AMFG'07 Proceedings of the 3rd international conference on Analysis and modeling of faces and gestures
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper addresses the recovery of face models from stereo pairs of images in the presence of foreign-body occlusions. In the proposed approach, a 3D morphable model (3DMM) for faces is augmented by an occlusion map defined on the model shape, and occlusion is detected with minimal computational overhead by incorporating robust estimators in the fitting process. Additionally, the method uses an explicit model for texture (or reflectance) in addition to shape, which is in contrast to most existing multi-view methods that use a shape model alone. We argue that both model components are required to handle certain classes of occluders, and we present empirical results to support this claim. In fact, the empirical results in this paper suggest that even in the absence of occlusions, stereo reconstruction using existing shape-only face models can perform poorly by some measures, and that the inclusion of an explicit texture model may be worth its computational expense.