Finding Actions Using Shape Flows

  • Authors:
  • Hao Jiang;David R. Martin

  • Affiliations:
  • Computer Science Department, Boston College, Chestnut Hill, USA MA 02467;Computer Science Department, Boston College, Chestnut Hill, USA MA 02467

  • Venue:
  • ECCV '08 Proceedings of the 10th European Conference on Computer Vision: Part II
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

We propose a novel method for action detection based on a new action descriptor called a shape flow that represents both the shape and movement of an object in a holistic and parsimonious manner. We find actions by finding shape flows in a target video that are similar to a template shape flow. Shape flows are largely independent of appearance, and the match cost function that we propose is invariant to scale changes and smooth nonlinear deformation in space and time. The problem of matching shape flows is difficult, however, yielding a large, non-convex, integer program. We propose a novel relaxation method based on successive convexification that converts this hard program into a vastly smaller linear program: By using only those variables that appear on the 4D lower convex hull of the matching cost volume, most of the variables in the linear program may be eliminated. Experiments confirm that the proposed shape flow method can successfully detect complex actions in cluttered video, even with self-occlusion, camera motion, and intra-class variation.