Explicit and implicit concept-based video retrieval with bipartite graph propagation model

  • Authors:

  • Lei Bao;Juan Cao;Yongdong Zhang;Jintao Li;Ming-yu Chen;Alexander G. Hauptmann

  • Affiliations:

  • Laboratory for Advanced Computing Technology Research, ICT, CAS, Beijing , China;Laboratory for Advanced Computing Technology Research, ICT, CAS, Beijing, China;Laboratory for Advanced Computing Technology Research, ICT, CAS, Beijing, China;Laboratory for Advanced Computing Technology Research, ICT, CAS, Beijing, China;School of Computer Science, Carnegie Mellon University, Pittsburgh, USA;School of Computer Science, Carnegie Mellon University, Pittsburgh, USA

  • Venue:
  • Proceedings of the international conference on Multimedia
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

The major scientific problem for content-based video retrieval is the semantic gap. Generally speaking, there are two appropriate ways to bridge the semantic gap: the first one is from human perspective (top-down) and the other one is from computer perspective (bottom-up). The top-down method defines a concept lexicon from human perspective, trains the detector for each concept based on supervised learning, and then indexes the corpus with concept detectors. Since each concept has an explicit semantic meaning, we call this concept as an explicit concept. The bottom-up approach directly discovers the underlying latent topics from video corpus by machine perspective using an unsupervised learning. The video corpus is indexed subsequently by these latent topics. As opposite to explicit concepts, we name latent topics as implicit concepts. Given the explicit concept set is pre-defined and independent of the corpus, it is impossible to completely describe corpus and users' queries. On the other hand, the implicit concepts are dynamic and dependent on the corpus, which is able to fully describe corpus and users' queries. Therefore, combining explicit and implicit concepts could be a promising way to bridge the semantic gap effectively. In this paper, a Bipartite Graph Propagation Model (BGPM) is applied to automatically balance influences from explicit and implicit concepts. Concept nodes with strong connections to queries are reinforced no matter explicit or implicit. Demonstrated by the experiments on TREVID 2008 video dataset, BGPM successfully fuses explicit and implicit concepts to achieve a significant improvement on 48 search tasks.