Multimodal metadata fusion using causal strength

  • Authors:
  • Yi Wu;Edward Y. Chang;Belle L. Tseng

  • Affiliations:
  • University of California, Santa Barbara, CA;University of California, Santa Barbara, CA;NEC Labs America, Cupertino, CA

  • Venue:
  • Proceedings of the 13th annual ACM international conference on Multimedia
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

We propose a probabilistic framework that uses influence diagrams to fuse metadata of multiple modalities for photo annotation. We fuse contextual information (location, time, and camera parameters), visual content (holistic and local perceptual features), and semantic ontology in a synergistic way. We use causal strengths to encode causalities between variables, and between variables and semantic labels. Through analytical and empirical studies, we demonstrate that our fusion approach can achieve high-quality photo annotation and good interpretability, substantially better than traditional methods.