Understanding dynamic scenes based on human sequence evaluation

  • Authors:
  • Jordi Gonzílez;Daniel Rowe;Javier Varona;F. Xavier Roca

  • Affiliations:
  • Institut de Robòtica i Informítica Industrial (UPC-CSIC), Edifici U, Parc Tecnològic de Barcelona, Barcelona, Catalonia, Spain;Computer Vision Center & Department of Computer Sciences (UAB), Edifici O, Campus UAB, Bellaterra, Catalonia, Spain;Unitat de Grífics i Visió per Ordinador (UIB), Edifici Anselm Turmeda, Campus UIB, Palma de Mallorca, Spain;Computer Vision Center & Department of Computer Sciences (UAB), Edifici O, Campus UAB, Bellaterra, Catalonia, Spain

  • Venue:
  • Image and Vision Computing
  • Year:
  • 2009

Quantified Score

Hi-index 0.02

Visualization

Abstract

In this paper, a Cognitive Vision System (CVS) is presented, which explains the human behaviour of monitored scenes using natural-language texts. This cognitive analysis of human movements recorded in image sequences is here referred to as Human Sequence Evaluation (HSE) which defines a set of transformation modules involved in the automatic generation of semantic descriptions from pixel values. In essence, the trajectories of human agents are obtained to generate textual interpretations of their motion, and also to infer the conceptual relationships of each agent w.r.t. its environment. For this purpose, a human behaviour model based on Situation Graph Trees (SGTs) is considered, which permits both bottom-up (hypothesis generation) and top-down (hypothesis refinement) analysis of dynamic scenes. The resulting system prototype interprets different kinds of behaviour and reports textual descriptions in multiple languages.