Learning Optical Flow

  • Authors:

  • Deqing Sun;Stefan Roth;J. P. Lewis;Michael J. Black

  • Affiliations:

  • Department of Computer Science, Brown University, Providence, USA;Department of Computer Science, TU Darmstadt, Darmstadt, Germany;Weta Digital Ltd., New Zealand;Department of Computer Science, Brown University, Providence, USA

  • Venue:
  • ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part III
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Assumptions of brightness constancy and spatial smoothness underlie most optical flow estimation methods. In contrast to standard heuristic formulations, we learn a statistical model of both brightness constancy error and the spatial properties of optical flow using image sequences with associated ground truth flow fields. The result is a complete probabilistic model of optical flow. Specifically, the ground truth enables us to model how the assumption of brightness constancy is violated in naturalistic sequences, resulting in a probabilistic model of "brightness inconstancy". We also generalize previous high-order constancy assumptions, such as gradient constancy, by modeling the constancy of responses to various linear filters in a high-order random field framework. These filters are free variables that can be learned from training data. Additionally we study the spatial structure of the optical flow and how motion boundaries are related to image intensity boundaries. Spatial smoothness is modeled using a Steerable Random Field, where spatial derivatives of the optical flow are steered by the image brightness structure. These models provide a statistical motivation for previous methods and enable the learning of all parameters from training data. All proposed models are quantitatively compared on the Middlebury flow dataset.