Human-centered attention models for video summarization

  • Authors:

  • Kaiming Li;Lei Guo;Carlos Faraco;Dajiang Zhu;Fan Deng;Tuo Zhang;Xi Jiang;Degang Zhang;Hanbo Chen;Xintao Hu;Stephen Miller;Tianming Liu

  • Affiliations:

  • Northwestern Polytechnical Univerisity, China and The University of Georgia;The University of Georgia;The University of Georgia;Northwestern Polytechnical Univerisity, China;Northwestern Polytechnical Univerisity, China;Northwestern Polytechnical Univerisity, China;The University of Georgia;Northwestern Polytechnical Univerisity, China;The University of Georgia;Northwestern Polytechnical Univerisity, China;The University of Georgia;The University of Georgia

  • Venue:
  • International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

A variety of user attention models for video/audio streams have been developed for video summarization and abstraction, in order to facilitate efficient video browsing and indexing. Essentially, human brain is the end user and evaluator of multimedia content and representation, and its responses can provide meaningful guidelines for multimedia stream summarization. For example, video/audio segments that significantly activate the visual, auditory, language and working memory systems of the human brain should be considered more important than others. It should be noted that user experience studies could be useful for such evaluations, but are suboptimal in terms of their capability of accurately capturing the full-length dynamics and interactions of the brain's response. This paper presents our preliminary efforts in applying the brain imaging technique of functional magnetic resonance imaging (fMRI) to quantify and model the dynamics and interactions between multimedia streams and brain response, when the human subjects are presented with the multimedia clips, in order to develop human-centered attention models that can be used to guide and facilitate more effective and efficient multimedia summarization. Our initial results are encouraging.