Collaborative multimodal photo annotation over digital paper

  • Authors:
  • Paulo Barthelmess;Edward Kaiser;Xiao Huang;David McGee;Philip Cohen

  • Affiliations:
  • Natural Interaction Systems, Seattle, WA;Natural Interaction Systems, Seattle, WA;Natural Interaction Systems, Seattle, WA;Natural Interaction Systems, Seattle, WA;Natural Interaction Systems, Seattle, WA

  • Venue:
  • Proceedings of the 8th international conference on Multimodal interfaces
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

The availability of metadata annotations over media content such as photos is known to enhance retrieval and organization, particularly for large data sets. The greatest challenge for obtaining annotations remains getting users to perform the large amount of tedious manual work that is required. In this demo we show a system for semi-automated labeling based on extraction of metadata from naturally occurring conversations of groups of people discussing pictures among themselves. The system supports a variety of collaborative label elicitation scenarios mixing co-located and distributed participants, operating primarily via speech, handwriting and sketching over tangible digital paper photo printouts. We demonstrate the real-time capabilities of the system by providing hands-on annotation experience for conference participants. Demo annotations are performed over public domain pictures portraying mainstream themes (e.g. from famous movies).