Recursive 3-D Visual Motion Estimation Using Subspace Constraints

• Authors: Stefano Soatto;Pietro Perona
• Venue: International Journal of Computer Vision
• Year: 1997

The 3-D motion of a camera within a static environment produces asequence of time-varying images that can be used for reconstructingthe relative motion between the scene and the viewer. The problem ofreconstructing rigid motion from a sequence of perspective images maybe characterized as the estimation of the state of a nonlineardynamical system, which is defined by the rigidity constraint and theperspective measurement map. The time-derivative of the measuredoutput of such a system, which is called the “2-D motion field” andis approximated by the “optical flow”, is bilinear in the motionparameters, and may be used to specify a subspace constraint on thedirection of heading independent of rotation and depth, and apseudo-measurement for the rotational velocity as a function of theestimated heading. The subspace constraint may be viewed as animplicit dynamical model with parameters on a differentiable manifold,and the visual motion estimation problem may be cast in asystem-theoretic framework as the identification of such animplicit model. We use techniques which pertain to nonlinearestimation and identification theory to recursively estimate 3-D rigidmotion from a sequence of images independent of the structure of thescene. Such independence from scene-structure allows us to deal with avariable number of visible feature-points and occlusions in aprincipled way. The further decoupling of the direction of headingfrom the rotational velocity generates a filter with a state thatbelongs to a two-dimensional and highly constrained state-space. As aresult, the filter exhibits robustness properties which arehighlighted in a series of experiments on real and noisy syntheticimage sequences. While the position of feature-points is not part ofthe state of the model, the innovation process of the filter describeshow each feature is compatible with a rigid motion interpretation,which allows us to test for outliers and makes the filter robust withrespect to errors in the feature tracking/optical flow, reflections,T-junctions. Once motion has been estimated, the 3-D structure of thescene follows easily. By releasing the constraint that the visiblepoints lie in front of the viewer, one may explain some psychophysicaleffects on the nonrigid percept of rigidly moving objects.