Robust subspace clustering by combined use of kNND metric and SVD algorithm
CVPR'04 Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition
| Hi-index | 0.00 |
A layer is a 2D sub-image inside which pixels share common apparent motion of some 3D scene plane. Representing videos with layers has many important applications, such as video compression, 3D scene and motion analysis, object detection and tracking, and vehicle navigation. Extracting layers from videos involves solving three highly intertwined subproblems: (1) segment the image into sub-regions (layers); (2) estimate the 2D motion of each layer; and (3) determine the number of layers. These three sub-problems are highly coupled together, making the layer extraction problem a very challenging one. Existing approaches to layer extraction are limited by (1) requiring good initial segmentation and/or strong assumptions about the scene, (2) unable to fully and simultaneously utilize the spatial and temporal constraints in video, and/or (3) unstable clustering in high dimensional space. This thesis presents a subspace approach to layer extraction which does not have the above limitations. We first show that the homographies induced by the planar patches in the scene form a linear subspace whose dimension is as low as two or three in many applications. We then formulate the layer extraction problem as clustering in such low dimensional subspace. Each layer in the input images will form a well-defined cluster in the subspace, and a simple mean shift based clustering algorithm can reliably identify the clusters thus the layers. A proof is presented to show that the subspace approach is guaranteed to increase significantly the layer discriminability, due to its ability to simultaneously utilize spatial and temporal constraints in the video. We present the detailed robust algorithm for layer extraction using subspace, as well as experimental results on a variety of real image sequences.