Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment

  • Authors:

  • Heung-Yeung Shum;Richard Szeliski

  • Affiliations:

  • Microsoft Research, One Microsoft Way, Redmond, WA 98052, USA. hshum@microsoft.com;Microsoft Research, One Microsoft Way, Redmond, WA 98052, USA. szeliski@microsoft.com

  • Venue:
  • International Journal of Computer Vision
  • Year:
  • 2000

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper presents a complete system forconstructing panoramic image mosaics from sequences of images.Our mosaic representation associates a transformation matrix witheach input image, rather than explicitly projecting all of the imagesonto a common surface (e.g., a cylinder). In particular, toconstruct a full view panorama, we introduce a rotationalmosaic representation that associates a rotation matrix (andoptionally a focal length) with each input image. A patch-basedalignment algorithm is developed to quickly align two images givenmotion models. Techniques for estimating and refining camera focallengths are also presented.In order to reduce accumulated registration errors, we applyglobal alignment (block adjustment) to the whole sequence of images,which results in an optimally registered image mosaic.To compensate for small amounts of motion parallax introduced bytranslations of the camera and other unmodeled distortions, we use alocal alignment (deghosting) technique which warps each image basedon the results of pairwise local image registrations. By combining bothglobal and local alignment, we significantly improve the quality ofour image mosaics, thereby enabling the creation of full viewpanoramic mosaics with hand-held cameras.We also present an inverse texture mapping algorithm forefficiently extracting environment mapsfrom our panoramic image mosaics. By mapping the mosaic onto an arbitrarytexture-mapped polyhedron surrounding the origin, we can explore thevirtual environment using standard 3D graphics viewers and hardware withoutrequiring special-purpose players.