Shape and motion from image streams under orthography: a factorization method
International Journal of Computer Vision
Non-parametric local transforms for computing visual correspondence
ECCV '94 Proceedings of the third European conference on Computer Vision (Vol. II)
An active vision architecture based on iconic representations
Artificial Intelligence - Special volume on computer vision
EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation
International Journal of Computer Vision
Efficient Region Tracking With Parametric Models of Geometry and Illumination
IEEE Transactions on Pattern Analysis and Machine Intelligence
Ordinal Measures for Visual Correspondence
CVPR '96 Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96)
Ranklets: Orientation Selective Non-Parametric Features Applied to Face Detection
ICPR '02 Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3 - Volume 3
A nonparametric approach to face detection using ranklets
AVBPA'03 Proceedings of the 4th international conference on Audio- and video-based biometric person authentication
Extracting Average Shapes from Occluded Non-rigid Motion
IbPRIA '07 Proceedings of the 3rd Iberian conference on Pattern Recognition and Image Analysis, Part II
Non-rigid metric reconstruction from perspective cameras
Image and Vision Computing
2D-3D registration of deformable shapes with manifold projection
ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
Fast algorithms for the computation of Ranklets
ICIP'09 Proceedings of the 16th IEEE international conference on Image processing
Successively alternate least square for low-rank matrix factorization with bounded missing data
Computer Vision and Image Understanding
Piecewise quadratic reconstruction of non-rigid surfaces from monocular sequences
ECCV'10 Proceedings of the 11th European conference on Computer vision: Part IV
Reconstruction of non-rigid 3D shapes from stereo-motion
Pattern Recognition Letters
SAMT'10 Proceedings of the 5th international conference on Semantic and digital media technologies
Optimal Metric Projections for Deformable and Articulated Structure-from-Motion
International Journal of Computer Vision
Direct Model-Based Tracking of 3D Object Deformations in Depth and Color Video
International Journal of Computer Vision
Adaptive Non-rigid Registration and Structure from Motion from Image Trajectories
International Journal of Computer Vision
Combining topological and view-based features for 3D model retrieval
Multimedia Tools and Applications
Recursive non-rigid structure from motion with online learned shape prior
Computer Vision and Image Understanding
Hi-index | 0.00 |
In this paper, we address the problem of estimating the 3D structure and motion of a deformable object given a set of image features tracked automatically throughout a video sequence. Our contributions are twofold: firstly, we propose a new approach to improve motion and structure estimates using a non-linear optimization scheme and secondly we propose a tracking algorithm based on ranklets, a recently developed family of orientation selective rank features. It has been shown that if the 3D deformations of an object can be modeled as a linear combination of shape bases then both its motion and shape may be recovered using an extension of Tomasi and Kanade's factorization algorithm for affine cameras. Crucially, these new factorization methods are model free and work purely from video in an unconstrained case: a single uncalibrated camera viewing an arbitrary 3D surface which is moving and articulating. The main drawback of existing methods is that they do not provide correct structure and motion estimates: the motion matrix has a repetitive structure which is not respected by the factorization algorithm. In this paper, we present a non-linear optimization method to refine the motion and shape estimates which minimizes the image reprojection error and imposes the correct structure onto the motion matrix by choosing an appropriate parameterization. Factorization algorithms require as input a set of feature tracks or correspondences found throughout the image sequence. The challenge here is to track the features while the object is deforming and the appearance of the image therefore changing. We propose a model free tracking algorithm based on ranklets, a multi-scale family of rank features that present an orientation selectivity pattern similar to Haar wavelets. A vector of ranklets is used to encode an appearance based description of a neighborhood of each tracked point. Robustness is enhanced by adapting, for each point, the shape of the filters to the structure of the particular neighborhood. A stack of models is maintained for each tracked point in order to manage large appearance variations with limited drift. Our experiments on sequences of a human subject performing different facial expressions show that this tracker provides a good set of feature correspondences for the non-rigid 3D reconstruction algorithm.