Measuring semantic coverage

  • Authors:
  • Sergei Nirenburg;Kavi Mahesh;Stephen Beale

  • Affiliations:
  • New Mexico State University, Las Cruces, NM;New Mexico State University, Las Cruces, NM;New Mexico State University, Las Cruces, NM

  • Venue:
  • COLING '96 Proceedings of the 16th conference on Computational linguistics - Volume 1
  • Year:
  • 1996

Quantified Score

Hi-index 0.00

Visualization

Abstract

The development of natural language processing systems is currently driven to a large extent by measures of knowledge-base size and coverage of individual phenomena relative to a corpus. While these measures have led to significant advances for knowledge-lean applications, they do not adequately motivate progress in computational semantics leading to the development of large-scale, general purpose NLP systems. In this article, we argue that depth of semantic representation is essential for covering a broad range of phenomena in the computational treatment of language and propose depth as an important additional dimension for measuring the semantic coverage of NLP systems. We propose an operationalization of this measure and show how to characterize an NLP system along the dimensions of size, corpus coverage, and depth. The proposed framework is illustrated using several prominent NLP systems. We hope the preliminary proposals made in this article will lead to prolonged debates in the field and will continue to be refined.