Scene aligned pooling for complex video recognition

  • Authors:

  • Liangliang Cao;Yadong Mu;Apostol Natsev;Shih-Fu Chang;Gang Hua;John R. Smith

  • Affiliations:

  • IBM T. J. Watson Research Center;Dept. Electrical Engineering, Columbia University;Google Research;Dept. Electrical Engineering, Columbia University;Dept. Computer Science, Stevens Institute of Technology;IBM T. J. Watson Research Center

  • Venue:
  • ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part II
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Real-world videos often contain dynamic backgrounds and evolving people activities, especially for those web videos generated by users in unconstrained scenarios. This paper proposes a new visual representation, namely scene aligned pooling, for the task of event recognition in complex videos. Based on the observation that a video clip is often composed with shots of different scenes, the key idea of scene aligned pooling is to decompose any video features into concurrent scene components, and to construct classification models adaptive to different scenes. The experiments on two large scale real-world datasets including the TRECVID Multimedia Event Detection 2011 and the Human Motion Recognition Databases (HMDB) show that our new visual representation can consistently improve various kinds of visual features such as different low-level color and texture features, or middle-level histogram of local descriptors such as SIFT, or space-time interest points, and high level semantic model features, by a significant margin. For example, we improve the-state-of-the-art accuracy on HMDB dataset by 20% in terms of accuracy.