Understanding natural language with diagrams

  • Authors:
  • Gordon S. Novak, Jr.;William C. Bulko

  • Affiliations:
  • Department of Computer Sciences, University of Texas at Austin, Austin, Texas;IBM Corporation, Austin, Texas

  • Venue:
  • AAAI'90 Proceedings of the eighth National conference on Artificial intelligence - Volume 1
  • Year:
  • 1990

Quantified Score

Hi-index 0.00

Visualization

Abstract

We describe a program, BEATRIX, that can understand textbook physics problems specified by a combination of English text and a diagram. The result of the understanding process is a unified internal model that represents the problem, including information derived from both the English text and the diagram. The system is implemented as two opportunistic coparsers, one for English and one for diagrams, within a blackboard architecture. A central problem is establishing coreference, that is, determining when parts of the text and diagram refer to the same object. Constraints supplied by the text and diagram mutually reduce ambiguity in interpretation of the other modality.