Technical Section: Realistic cloth augmentation in single view video under occlusions

  • Authors:

  • Anna Hilsmann;David C. Schneider;Peter Eisert

  • Affiliations:

  • Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institute, Einsteinufer 37, 10587 Berlin, Germany;Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institute, Einsteinufer 37, 10587 Berlin, Germany;Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institute, Einsteinufer 37, 10587 Berlin, Germany and Humboldt University Berlin, Department of Computer Science, Unter den Linden 6, 10 ...

  • Venue:
  • Computers and Graphics
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

Augmenting cloth in real video is a challenging task because cloth performs complex motions and deformations and produces complex shading on the surface. Therefore, for a realistic augmentation of cloth, parameters describing both deformation as well as shading properties are needed. Furthermore, objects occluding the real surface have to be taken into account as on the one hand they affect the parameter estimation and on the other hand should also occlude the virtually textured surface. This is especially challenging in monocular image sequences where a 3-dimensional reconstruction of complex surfaces is difficult to achieve. In this paper, we present a method for cloth retexturing in monocular image sequences under external occlusions without a reconstruction of the 3-dimensional geometry. We exploit direct image information and simultaneously estimate deformation and photometric parameters using a robust estimator which detects occluded pixels as outliers. Additionally, we exploit the estimated parameters to establish an occlusion map from local statistical color models of texture surface patches that are established during tracking. With this information we can produce convincing augmented results.