View-invariant Alignment and Matching of Video Sequences

• Authors: Cen Rao;Alexei Gritai;Mubarak Shah;Tanveer Syeda-Mahmood
• Venue: ICCV '03 Proceedings of the Ninth IEEE International Conference on Computer Vision - Volume 2
• Year: 2003

In this paper, we propose a novel method to establish temporalcorrespondence between the frames of two videos. 3D epipolargeometry is used to eliminate the distortion generated bythe projection from 3D to 2D. Although the fundamental matrixcontains the extrinsic property of the projective geometrybetween views, it is sensitive to noise. Therefore, wepropose the use of a rank constraint of corresponding pointsin two views to measure the similarity between trajectories.This rank constraint shows more robustness and avoids computationof the fundamental matrix. A dynamic programmingapproach using the similarity measurement is proposed to findthe non-linear time-warping function for videos containinghuman activities. In this way, videos of different individualstaken at different times and from distinct viewpoints canbe synchronized. A temporal pyramid of trajectories is appliedto improve the accuracy of the view-invariant dynamictime-warping approach. We show various applications of thisapproach such as video synthesis, human action recognition,and computer aider training. Compared to state-of-the-arttechniques, our method shows a great improvement.