Learning Shape from Defocus

• Authors: Paolo Favaro;Stefano Soatto
• Venue: ECCV '02 Proceedings of the 7th European Conference on Computer Vision-Part II
• Year: 2002

We present a novel method for inferring three-dimensional shape from a collection of defocused images. It is based on the observation that defocused images are the null-space of certain linear operators that depend on the three-dimensional shape of the scene as well as on the optics of the camera. Unlike most current work based on inverting the imaging model to recover the "deblurred" image and the shape of the scene, we approach the problem from a new angle by collecting a number of deblurred images, and estimating the operator that spans their left null space directly. This is done using a singular value decomposition. Since the operator depends on the depth of the scene, we repeat the procedure for a number of different depths. Once this is done, depth can be recovered in real time: the new image is projected onto each null-space, and the depth that results in the smallest residual is chosen. The most salient feature of this algorithm is its robustness: not only can one learn the operators with one camera and then use them to successfully retrieve depth from images taken with another camera, but one can even learn the operators from simulated images, and use them to retrieve depth from real images. Thus we train the system with synthetic patterns, and then use it on real data without knowledge of the optics of the camera. Another attractive feature is that the algorithm does not rely on a discretization or an approximation of the radiance of the scene (the "deblurred" image). In fact, the operator we recover is finite-dimensional, but it arises as the orthogonal projector of a semiinfinite operator that maps square-integrable radiance distributions onto images. Thus, the radiance is never approximated or represented via a finite set of filters. Instead, the rank of the operator learned from real data provides an estimate of the intrinsic dimensionality of the radiance distribution of real images. The algorithm is optimal in the sense of L2 and can be implemented in real time.