Why did the person cross the road (there)? scene understanding using probabilistic logic models and common sense reasoning

  • Authors:
  • Aniruddha Kembhavi;Tom Yeh;Larry S. Davis

  • Affiliations:
  • University of Maryland, College Park;University of Maryland, College Park;University of Maryland, College Park

  • Venue:
  • ECCV'10 Proceedings of the 11th European conference on Computer vision: Part II
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

We develop a video understanding system for scene elements, such as bus stops, crosswalks, and intersections, that are characterized more by qualitative activities and geometry than by intrinsic appearance. The domain models for scene elements are not learned from a corpus of video, but instead, naturally elicited by humans, and represented as probabilistic logic rules within a Markov Logic Network framework. Human elicited models, however, represent object interactions as they occur in the 3D world rather than describing their appearance projection in some specific 2D image plane. We bridge this gap by recovering qualitative scene geometry to analyze object interactions in the 3D world and then reasoning about scene geometry, occlusions and common sense domain knowledge using a set of meta-rules. The effectiveness of this approach is demonstrated on a set of videos of public spaces.