Semantic event representation and recognition using syntactic attribute graph grammar

  • Authors:

  • Liang Lin;Haifeng Gong;Li Li;Liang Wang

  • Affiliations:

  • School of Information Science and Technology, Beijing Institute of Technology, Beijing 100081, China and Lotus Hill Research Institute for Computer Vision and Information Science, Ezhou 436000, Ch ...;Lotus Hill Research Institute for Computer Vision and Information Science, Ezhou 436000, China;Ocean University of China, Qingdao 266071, China;Institute of Computing Technology of CAS, Beijing 100190, China

  • Venue:
  • Pattern Recognition Letters
  • Year:
  • 2009

Quantified Score

Hi-index 0.10

Visualization

Abstract

The representation and recognition of complex semantic events (e.g. illegal parking, stealing objects) is a challenging task for high-level understanding of video sequence. To solve this problem, an attribute graph grammar for events modeling is studied in this paper. This grammar models the variability of semantic events by a set of meaningful ''event components'' with the spatio-temporal constraints. The event components are defined manually according to their semantic meaning, and further decomposed into atomic event primitives. These event primitives are learned on a object-trajectory table that describes mobile object attributes (location, velocity, and visibility) in a video sequence. A dictionary of temporal and spatial relations are defined to constrain the event primitives. With this representation, one observed event can be parsed into an ''event parse graph'', and all possible variability of one event can be modeled into an ''event And-Or graph'', in a syntactic way. The probability model of an ''event And-Or graph'' can be learned on a set of annotated event instances, and given a learned event And-Or graph, a Gibbs sampling scheme is utilized for inference on a testing video. In the experiments, we test events recognition performance of the proposed on both real indoor and outdoor videos and show quantitative recognition rate on the public LHI dataset.