Predicting 3d people from 2d pictures

  • Authors:
  • Leonid Sigal;Michael J. Black

  • Affiliations:
  • Department of Computer Science, Brown University, Providence, RI;Department of Computer Science, Brown University, Providence, RI

  • Venue:
  • AMDO'06 Proceedings of the 4th international conference on Articulated Motion and Deformable Objects
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

We propose a hierarchical process for inferring the 3D pose of a person from monocular images. First we infer a learned view-based 2D body model from a single image using non-parametric belief propagation. This approach integrates information from bottom-up body-part proposal processes and deals with self-occlusion to compute distributions over limb poses. Then, we exploit a learned Mixture of Experts model to infer a distribution of 3D poses conditioned on 2D poses. This approach is more general than recent work on inferring 3D pose directly from silhouettes since the 2D body model provides a richer representation that includes the 2D joint angles and the poses of limbs that may be unobserved in the silhouette. We demonstrate the method in a laboratory setting where we evaluate the accuracy of the 3D poses against ground truth data. We also estimate 3D body pose in a monocular image sequence. The resulting 3D estimates are sufficiently accurate to serve as proposals for the Bayesian inference of 3D human motion over time