A dataset and evaluation methodology for visual saliency in video

  • Authors:

  • Jia Li;Yonghong Tian;Tiejun Huang;Wen Gao

  • Affiliations:

  • Key Lab of Intell. Info. Process, Inst. of Comput. Tech., Chinese Academy of Sciences and Graduate University of Chinese Academy of Sciences, China;Institute of Digital Media, School of EE & CS, Peking University, China;Institute of Digital Media, School of EE & CS, Peking University, China;Key Lab of Intell. Info. Process, Inst. of Comput. Tech., Chinese Academy of Sciences and Institute of Digital Media, School of EE & CS, Peking University, China

  • Venue:
  • ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

Recently, visual saliency has drawn great research interest in the field of computer vision and multimedia. Various approaches aiming at calculating visual saliency have been proposed. To evaluate these approaches, several datasets have been presented for visual saliency in images. However, there are few datasets to capture spatiotemporal visual saliency in video. Intuitively, visual saliency in video is strongly affected by temporal context and might vary significantly even in visually similar frames. In this paper, we present an extensive dataset with 7.5-hour videos to capture spatiotemporal visual saliency. The salient regions in frames sequentially sampled from these videos are manually labeled by 23 subjects and then averaged to generate the ground-truth saliency maps. We also present three metrics to evaluate competing approaches. Several typical algorithms were evaluated on the dataset. The experimental results show that this dataset is very suitable for evaluating visual saliency. We also discover some interesting findings that would be addressed in future research. Currently, the dataset is freely available online together with the source code for evaluation.