Joint people, event, and location recognition in personal photo collections using cross-domain context

  • Authors:
  • Dahua Lin;Ashish Kapoor;Gang Hua;Simon Baker

  • Affiliations:
  • Computer Science and Artificial Intelligence Laboratory, MIT and Microsoft Research;Microsoft Research;Nokia Research Center Hollywood;Microsoft Research

  • Venue:
  • ECCV'10 Proceedings of the 11th European conference on Computer vision: Part I
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a framework for vision-assisted tagging of personal photo collections using context. Whereas previous efforts mainly focus on tagging people, we develop a unified approach to jointly tag across multiple domains (specifically people, events, and locations). The heart of our approach is a generic probabilistic model of context that couples the domains through a set of cross-domain relations. Each relation models how likely the instances in two domains are to co-occur. Based on this model, we derive an algorithm that simultaneously estimates the cross-domain relations and infers the unknown tags in a semi-supervised manner. We conducted experiments on two well-known datasets and obtained significant performance improvements in both people and location recognition. We also demonstrated the ability to infer event labels with missing timestamps (i.e. with no event features).