Variational methods for shape reconstruction in computer vision

• Authors: Hailin Jin;Stefano Soatto
• Venue: Variational methods for shape reconstruction in computer vision
• Year: 2003

This dissertation addresses the problem of inferring the three-dimensional shape and radiance of a scene from a collection of calibrated images taken from different viewpoints. The problem is known as “multi-view stereo” and has been studied extensively in the Computer Vision literature. Most existing multi-view stereo algorithms rely on identification of corresponding points or regions among images. They work effectively only when scenes are Lambertian and have textured radiances, because lack of Lambertianity or absence of texture in the radiance leads to an ill-posed correspondence problem. Targeting scenes with more complex photometric properties, we propose a framework of comparing the images with a model of the scene, which avoids the ill-posed correspondence problem. The model is comprised of a component for shape and a component for radiance. We model the shape as a collection of smooth surfaces and model the radiance based on the underlying reflectance properties. Within this framework, we present two stereo reconstruction algorithms, one for non-Lambertian scenes and the other for Lambertian scenes with smooth radiances. In addressing non-Lambertian scenes, we propose a novel model for the radiance. At the core of the model lies a rank constraint of the radiance tensor field and a discrepancy function that measures how well the image data satisfy the constraint. We exploit this model to reconstruct both the shape and radiance. The latter can be used to generate novel images that preserve the non-Lambertian appearance of the scene. In addressing Lambertian scenes with smooth radiances, we propose to model the radiance as a smooth function defined on an unknown surface. We then estimate both the shape and radiance by optimizing a cost functional which combines a geometric prior on shape, a smoothness prior on radiance and a data fitness score. Our algorithm jointly segments images in a region-based fashion. This scheme does not require computing image gradients and therefore is robust to image noise.