Taxonomy of directing semantics for film shot classification

  • Authors:

  • Hee Lin Wang;Loong-Fah Cheong

  • Affiliations:

  • Institute of Infocomm Research, Agency for Science, Technology and Research, Singapore;Department of Electrical and Computer Engineering, National University of Singapore, Singapore

  • Venue:
  • IEEE Transactions on Circuits and Systems for Video Technology
  • Year:
  • 2009

Quantified Score

Hi-index 0.03

Visualization

Abstract

The immense indexing potential of motion cues has hitherto been realized only in domains with more apparent structure (e.g., sport videos). To address the lack of theoretical attention and to realize the potential of motion-based indexing in the subtler film domain, we propose a systematic approach to build taxonomy for film directing semantics. These motionrelated semantics are grounded upon cinematography and are thus more appealing to users. In order to automate the classification of these semantics, we have developed a novel markov random field based motion segmentation algorithm with an integral foreground/background identification capability based on edge occlusion reasoning. This algorithm is sufficiently robust and fast for film domain conditions, and allows us to formulate salient and novel motion descriptors capable of mapping to the proposed directing semantics. We demonstrate the validity of the framework and effectiveness of the motion-based descriptors by classifying shots from Hollywood domain movies according to the proposed taxonomy with satisfactory results.