Annotating Images by Mining Image Search Results

  • Authors:

  • Xin-Jing Wang;Lei Zhang;Xirong Li;Wei-Ying Ma

  • Affiliations:

  • Microsoft Research Asia, Beijing;Microsoft Research Asia, Beijing;University of Amsterdam, Amsterdam;Microsoft Research Asia, Beijing

  • Venue:
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Year:
  • 2008

Quantified Score

Hi-index 0.14

Visualization

Abstract

In this paper, we propose a novel attempt of model-free image annotation which annotates images by mining their search results. It contains three steps: 1) the search process to discover visually and semantically similar search results; 2) the mining process to identify salient terms from textual descriptions of the search results; and 3) the annotation rejection process to filter out noisy terms yielded by step 2). To ensure real time annotation, two key techniques are leveraged - one is to map the high dimensional image visual features into hash codes, the other is to implement it as a distributed system, of which the search and mining processes are provided as Web services. As a typical result, the entire process finishes in less than 1 second. Our proposed approach enables annotating with unlimited vocabulary, and is highly scalable and robust to outliers. Experimental results on both real web images and a bench mark image dataset show the effectiveness and efficiency of the proposed algorithm.