Screw-transform manifolds for camera self calibration

• Authors: Russell Alan Manning;Charles R. Dyer
• Venue: Screw-transform manifolds for camera self calibration
• Year: 2003

This dissertation concerns the mathematical theory of screw-transform manifolds and their use in camera self calibration. A camera's calibration is the function that maps 3D scene points to 2D image points, e.g., in photographs taken by the camera. Between every two photographs taken from different positions there exists a pairwise constraint called the “fundamental matrix,” which can be computed directly from the images. When the two photographs are captured by the same camera, the fundamental matrix induces a surface in calibration space called a “screw-transform manifold.” This manifold represents every possible internal calibration for the camera. By acquiring several different pairwise fundamental matrices, several different screw-transform manifolds can be computed; however, the internal calibration of the original camera must be a member of each manifold and hence, by finding the intersection point of all the manifolds, the camera's calibration can be determined. The process of determining calibration directly from images taken by a camera is called “self calibration.” The contributions of this dissertation include the theory of screw-transform manifolds and three original algorithms for determining the mutual intersection points of a collection of manifolds. While many papers have been written on self calibration, almost all previous methods posed their solutions as the global minima of an error function. However, performing global optimization is problematic; it is easy to locate a local minimum without finding the global minimum, and in some cases the attraction basin of the global minimum is so small that the algorithm must essentially “guess” the solution in order to find it. One of the new approaches created as part of this dissertation, called STM-SURFIT, avoids global optimization altogether and can effectively locate all global minima in a single pass, running in under 1 second on a modern home computer. The general approach used to avoid the problems of optimization may have wider applicability than simply camera calibration. A tutorial on multiview geometry that assumes only knowledge of linear algebra is included to provide the necessary mathematical background. The related history and previous work on self calibration and image-based rendering is also presented. As part of the theory of screw-transform manifolds, a theorem is introduced that partitions monocular view pairs into six categories based on the underlying screw motion of the camera and provides a simple test for determining category. In addition, some methods for self calibration and image-based rendering from dynamic scenes are presented. The image-based rendering techniques do not require camera calibration but are limited in applicability; this adds to the growing body of evidence that camera calibration is a necessity for most useful image-based rendering techniques.