Object fingerprints for content analysis with applications to street landmark localization

  • Authors:

  • Wen Wu;Jie Yang

  • Affiliations:

  • Carnegie Mellon University, Pittsburgh, PA, USA;Carnegie Mellon University, Pittsburgh, PA, USA

  • Venue:
  • MM '08 Proceedings of the 16th ACM international conference on Multimedia
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

An object can be a basic unit for multimedia content analysis. Besides similarity among common objects, each object has its own unique characteristics which we cannot find in other surrounding objects in multimedia data. We call such unique characteristics object fingerprints. In this paper, we propose a novel approach to extract and match object fingerprints for multimedia content analysis. In particular, we focus on the problem of street landmark localization from images. Instead of modeling and matching a street landmark as a whole, our proposed approach extracts the landmark's object fingerprints in a given image and match to a new image or video in order to localize the landmark. We formulate matching the landmark's object fingerprints as a classification problem solved by a cascade of 1NN classifiers. We develop a street landmark localization system that combines salient region detection, segmentation, and object fingerprint extraction techniques for the purpose. To evaluate, we have compiled a novel dataset which consists of 15 U.S. street landmarks' images and videos. Our experiments on this dataset show superior performance to state-of-the-art recognition algorithms [20, 33]. The proposed approach can also be well generalized to other objects of interest and content analysis tasks. We demonstrate the feasibility through the application of our approach to refine web image search results and obtained encouraging results.