Natural Language Description of Image Sequences as a Form of Knowledge Representation

  • Authors:
  • Hans-Hellmut Nagel

  • Affiliations:
  • -

  • Venue:
  • KI '99 Proceedings of the 23rd Annual German Conference on Artificial Intelligence: Advances in Artificial Intelligence
  • Year:
  • 1999

Quantified Score

Hi-index 0.00

Visualization

Abstract

An image sequence evaluation process combines information from different information sources. One of these sources is a camera which records a scene and provides the acquired information as a digitized image sequence. A different source provides knowledge regarding signal processing and geometry, exploited in order to map the image sequence signal to a system-internal representation of visible bodies and their movement in the depicted scene. Still another type of source provides abstract conceptual knowledge linking the system-internal geometric representation to tasks and goals of agents which act within the depicted scene or may influence it from the outside. Rather than providing this third type of information for inference engines by 'handcrafted' rules or sets of axioms, it is postulated that this type of knowledge should be derived by algorithmic analysis of a suitably formulated natural language text: natural language text is considered as a genuine represention of abstract knowledge for an image sequence evaluation process. This hypothesis is studied for the example of a system which transforms video sequences of road scenes into natural language text describing the recorded actual traffic.