Referring to world objects with text and pictures

  • Authors:
  • Elisabeth André;Thomas Rist

  • Affiliations:
  • German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany;German Research Center for Artificial Intelligence (DFKI), Saarbrücken, Germany

  • Venue:
  • COLING '94 Proceedings of the 15th conference on Computational linguistics - Volume 1
  • Year:
  • 1994

Quantified Score

Hi-index 0.00

Visualization

Abstract

It often makes sense to employ both text and pictures when referring to world objects. In this paper, we present a model for referring which is based on the assumption that concepts may be activated not only by text, but also by pictures and text-picture combinations. By means of a case study, we demonstrate that failure and success of referring acts can be explained by the user's ability to infer certain links between mental representations and object descriptions. Finally, we show how the model has been incorporated into a plan-based multimedia presentation system by defining operators for concept activation.