Automatic occlusion removal from facades for 3D urban reconstruction

  • Authors:
  • Chris Engels;David Tingdahl;Mathias Vercruysse;Tinne Tuytelaars;Hichem Sahli;Luc Van Gool

  • Affiliations:
  • K.U. Leuven, ESAT-PSI/IBBT;K.U. Leuven, ESAT-PSI/IBBT;K.U. Leuven, ESAT-PSI/IBBT;K.U. Leuven, ESAT-PSI/IBBT;V.U. Brussel, ETRO;K.U. Leuven, ESAT-PSI/IBBT and ETH Zurich, BIWI

  • Venue:
  • ACIVS'11 Proceedings of the 13th international conference on Advanced concepts for intelligent vision systems
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Object removal and inpainting approaches typically require a user to manually create a mask around occluding objects. While creating masks for a small number of images is possible, it rapidly becomes untenable for longer image sequences. Instead, we accomplish this step automatically using an object detection framework to explicitly recognize and remove several classes of occlusions. We propose using this technique to improve 3D urban reconstruction from street level imagery, in which building facades are frequently occluded by vegetation or vehicles. By assuming facades in the background are planar, 3D scene estimation provides important context to the inpainting process by restricting input sample patches to regions that are coplanar to the occlusion, leading to more realistic final textures. Moreover, because non-static and reflective occlusion classes tend to be difficult to reconstruct, explicitly recognizing and removing them improves the resulting 3D scene.