Home photo indexing using learned visual keywords

  • Authors:
  • Joo-Hwee Lim;Jesse S. Jin

  • Affiliations:
  • Institute of Infocomm Research, Singapore;The University of Sydney, Sydney, Australia

  • Venue:
  • VIP '02 Selected papers from the 2002 Pan-Sydney workshop on Visualisation - Volume 22
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

With rapid advances in sensor, storage, processor, and communication technologies, consumers can now afford to create, store, process, and share large digital photo collections. With more and more digital photos accumulated, consumers need effective and efficient tools to index and retrieve relevant photos. In this paper, we propose a novel image representation called Visual Keyword Histogram (VKH) for content-based indexing and retrieval. Visual keywords are domain-relevant visual prototypes (e.g. faces, foliage, buildings etc) with both perceptual appearance and textual semantics. Collectively, VKHs are computed over spatial tessellation to represent the distribution of visual keywords in various parts of an image. To construct a vocabulary of visual keywords, an incremental neural network is adopted to learn visual keywords from examples. This allows us to build domain-specific visual vocabularies rapidly and incrementally. We demonstrate our approach on 2400 home photos with 15 semantic queries.