Classification of semantic relations by humans and machines

  • Authors:
  • Erwin Marsi;Emiel Krahmer

  • Affiliations:
  • Tilburg University, The Netherlands;Tilburg University, The Netherlands

  • Venue:
  • EMSEE '05 Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment
  • Year:
  • 2005

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper addresses the classification of semantic relations between pairs of sentences extracted from a Dutch parallel corpus at the word, phrase and sentence level. We first investigate the performance of human annotators on the task of manually aligning dependency analyses of the respective sentences and of assigning one of five semantic relations to the aligned phrases (equals, generalizes, specifies, restates and intersects). Results indicate that humans can perform this task well, with an F-score of .98 on alignment and an F-score of .95 on semantic relations (after correction). We then describe and evaluate a combined alignment and classification algorithm, which achieves an F-score on alignment of .85 (using EuroWordNet) and an F-score of .80 on semantic relation classification.