Toward multimodal situated analysis

  • Authors:
  • Chreston Miller;Francis Quek

  • Affiliations:
  • Virginia Tech, Blacksburg, VA, USA;Virginia Tech, Blacksburg, VA, USA

  • Venue:
  • ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Multimodal analysis of human behavior is ultimately situated. The situated context of an instance of a behavior phenomenon informs its analysis. Starting with some initial (user-supplied) descriptive model of a phenomenon, accessing and studying instances in the data that are matches or near matches to the model is essential to refine the model to account for variations in the phenomenon. This inquiry requires viewing the instances within-context to judge their relevance. In this paper, we propose an automatic processing approach that supports this need for situated analysis in multimodal data. We process events on a semi-interval level to provide detailed temporal ordering of events with respect to instances of a phenomenon. We demonstrate the results of our approach and how it facilitates and allows for situated multimodal analysis.