Multimedia Evidence Fusion for Video Concept Detection via OWA Operator

  • Authors:

  • Ming Li;Yan-Tao Zheng;Shou-Xun Lin;Yong-Dong Zhang;Tat-Seng Chua

  • Affiliations:

  • Key Laboratory of Intelligent Information Processing, ICT, CAS, Beijing, China 100190 and Department of Computer Science, National University of Singapore, Singapore 117543 and Graduate School of ...;Department of Computer Science, National University of Singapore, Singapore 117543;Key Laboratory of Intelligent Information Processing, ICT, CAS, Beijing, China 100190;Key Laboratory of Intelligent Information Processing, ICT, CAS, Beijing, China 100190;Department of Computer Science, National University of Singapore, Singapore 117543

  • Venue:
  • MMM '09 Proceedings of the 15th International Multimedia Modeling Conference on Advances in Multimedia Modeling
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a novel multi-modal evidence fusion method for highlevel feature (HLF) detection in videos. The uni-modal features, such as color histogram, transcript texts, etc, tend to capture different aspects of HLFs and hence share complementariness and redundancy in modeling the contents of such HLFs. We argue that such inter-relation are key to effective multi-modal fusion. Here, we formulate the fusion as a multi-criteria group decision making task, in which the uni-modal detectors are coordinated for a consensus final detection decision, based on their inter-relations. Specifically, we mine the complementariness and redundancy inter-relation of uni-modal detectors using the Ordered Weighted Average (OWA) operator. The `or-ness' measure in OWA models the inter-relation of uni-modal detectors as combination of pure complementariness and pure redundancy. The resulting weights of OWA can then yield a consensus fusion, by optimally leveraging the decisions of uni-modal detectors. The experiments on TRECVID 07 dataset show that the proposed OWA aggregation operator can significantly outperform other fusion methods, by achieving a state-of-art MAP of 0.132.