Video2Sentence and vice versa

  • Authors:
  • Amirhossein Habibian;Cees G.M. Snoek

  • Affiliations:
  • Informatics Institute, University of Amsterdam, Amsterdam, Netherlands;Informatics Institute, University of Amsterdam, Amsterdam, Netherlands

  • Venue:
  • Proceedings of the 21st ACM international conference on Multimedia
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this technical demonstration, we showcase a multimedia search engine that retrieves a video from a sentence, or a sentence from a video. The key novelty is our machine translation capability that exploits a cross-media representation for both the visual and textual modality using concept vocabularies. We will demonstrate the translations using arbitrary web videos and sentences related to everyday events. What is more, we will provide an automatically generated explanation, in terms of concept detectors, on why a particular video or sentence has been retrieved as the most likely translation.