Vision and action in the language-ready brain: from mirror neurons to SemRep

  • Authors:
  • Michael A. Arbib;Jin Yong Lee

  • Affiliations:
  • Computer Science and Neuroscience and University of Southern California, Los Angeles, CA;Computer Science, University of Southern California, Los Angeles, CA

  • Venue:
  • BVAI'07 Proceedings of the 2nd international conference on Advances in brain, vision and artificial intelligence
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

The general setting for our work is to locate language perception and production within the broader context of brain mechanisms for action and perception in general, modeling brain function in terms of the competition and cooperation of schemas. Particular emphasis is placed on mirror neurons - neurons active both for execution of a certain class of actions and for recognition of a (possibly broader) class of similar actions. We build on the early VISIONS model of schema-based computer analysis of static scenes to present SemRep, a graphical representation of dynamic visual scenes designed to support the generation of varied descriptions of episodes. Mechanisms for parsing and production of sentences are currently being implemented within Template Construction Grammar (TCG), a new form of construction grammar distinguished by its use of SemRep to express semantics.