Latent semantic retrieval of personal photos with sparse user annotation by fused image/speech/text features

  • Authors:

  • Yi-sheng Fu; Chia-yu Wan; Lin-shan Lee

  • Affiliations:

  • Graduate Institute of Computer Science and Information Engineering, National Taiwan University, Taiwan;Graduate Institute of Communication Engineering, National Taiwan University, Taiwan;Graduate Institute of Computer Science and Information Engineering, National Taiwan University, Taiwan

  • Venue:
  • ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

While users prefer high-level semantic photo descriptions (e.g., who, what, when, where), we wish to minimize the need to annotate photos using such descriptions by the user. We propose a latent semantic personal photo retrieval approach using fused image/speech/text features. We use low-level image features to derive relatoionships among sparsely annotated photos, and probabilistic latent semantic analysis (PLSA) models based on fused image/speech/text features to analyze photo “topics”. We then retrieve the photos using text or speech queries of simple high-level semantic words only. In preliminary experiments, while only 10% of the photos were manually annotated, the photos could be well retrieved with very encouraging results.