Shape and motion from image streams under orthography: a factorization method
International Journal of Computer Vision
The Geometry of Algorithms with Orthogonality Constraints
SIAM Journal on Matrix Analysis and Applications
A Direct Method for 3D Factorization of Nonrigid Motion Observed in 2D
CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Damped Newton Algorithms for Matrix Factorization with Missing Data
CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Articulated Structure from Motion by Factorization
CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
Uncalibrated Perspective Reconstruction of Deformable Structures
ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision - Volume 2
A Closed-Form Solution to Non-Rigid Shape and Motion Recovery
International Journal of Computer Vision
Non-Rigid Metric Shape and Motion Recovery from Uncalibrated Images Using Priors
CVPR '06 Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1
Non-rigid structure from motion using ranklet-based tracking and non-linear optimization
Image and Vision Computing
Rotation constrained power factorization for structure from motion of nonrigid objects
Pattern Recognition Letters
Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors
IEEE Transactions on Pattern Analysis and Machine Intelligence
IEEE Transactions on Pattern Analysis and Machine Intelligence
Perspective Nonrigid Shape and Motion Recovery
ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part I
Estimating 3D shape from degenerate sequences with missing data
Computer Vision and Image Understanding
Optimal shape from motion estimation with missing and degenerate data
WMVC '08 Proceedings of the 2008 IEEE Workshop on Motion and video Computing
International Journal of Computer Vision
Group-Valued regularization for analysis of articulated motion
ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part I
| Hi-index | 0.00 |
This paper describes novel algorithms for recovering the 3D shape and motion of deformable and articulated objects purely from uncalibrated 2D image measurements using a factorisation approach. Most approaches to deformable and articulated structure from motion require to upgrade an initial affine solution to Euclidean space by imposing metric constraints on the motion matrix. While in the case of rigid structure the metric upgrade step is simple since the constraints can be formulated as linear, deformability in the shape introduces non-linearities. In this paper we propose an alternating bilinear approach to solve for non-rigid 3D shape and motion, associated with a globally optimal projection step of the motion matrices onto the manifold of metric constraints. Our novel optimal projection step combines into a single optimisation the computation of the orthographic projection matrix and the configuration weights that give the closest motion matrix that satisfies the correct block structure with the additional constraint that the projection matrix is guaranteed to have orthonormal rows (i.e. its transpose lies on the Stiefel manifold). This constraint turns out to be non-convex. The key contribution of this work is to introduce an efficient convex relaxation for the non-convex projection step. Efficient in the sense that, for both the cases of deformable and articulated motion, the proposed relaxations turned out to be exact (i.e. tight) in all our numerical experiments. The convex relaxations are semi-definite (SDP) or second-order cone (SOCP) programs which can be readily tackled by popular solvers. An important advantage of these new algorithms is their ability to handle missing data which becomes crucial when dealing with real video sequences with self-occlusions. We show successful results of our algorithms on synthetic and real sequences of both deformable and articulated data. We also show comparative results with state of the art algorithms which reveal that our new methods outperform existing ones.