Cross-media entity recognition in nearly parallel visual and textual documents

  • Authors:
  • Koen Deschacht;Marie-Francine Moens;Wouter Robeyns

  • Affiliations:
  • Katholieke Universiteit Leuven - Legal Informatics and Information Retrieval, Leuven, Belgium;Katholieke Universiteit Leuven - Legal Informatics and Information Retrieval, Leuven, Belgium;Katholieke Universiteit Leuven - Legal Informatics and Information Retrieval, Leuven, Belgium

  • Venue:
  • Large Scale Semantic Access to Content (Text, Image, Video, and Sound)
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a novel approach to automatically annotate images solely using associated text. We detect and classify all entities (persons and objects) in the text after which we determine the salience (the importance of an entity in a text) and visualness (the extent to which an entity can be perceived visually) of these entities. We combine these measures to compute the probability that an entity is present in the image. The suitability of our approach was successfully tested on 900 image-text pairs of Yahoo! News.