Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking

  • Authors:

  • Bodo Rosenhahn;Thomas Brox;Joachim Weickert

  • Affiliations:

  • Max-Planck-Center Saarbrücken, Saarbrücken, Germany 66123;Computer Vision and Pattern Recognition Group, University of Bonn, Bonn, Germany 53117;Mathematical Image Analysis Group, Saarland University, Saarbrücken, Germany 66041

  • Venue:
  • International Journal of Computer Vision
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this article we present the integration of 3-D shape knowledge into a variational model for level set based image segmentation and contour based 3-D pose tracking. Given the surface model of an object that is visible in the image of one or multiple cameras calibrated to the same world coordinate system, the object contour extracted by the segmentation method is applied to estimate the 3-D pose parameters of the object. Vice-versa, the surface model projected to the image plane helps in a top-down manner to improve the extraction of the contour. While common alternative segmentation approaches, which integrate 2-D shape knowledge, face the problem that an object can look very differently from various viewpoints, a 3-D free form model ensures that for each view the model can fit the data in the image very well. Moreover, one additionally solves the problem of determining the object's pose in 3-D space. The performance is demonstrated by numerous experiments with a monocular and a stereo camera system.