Connective-based measuring of the inter-annotator agreement in the annotation of discourse in PDT

  • Authors:

  • Jiří Mírovský;Lucie Mladová;Šárka Zikánová

  • Affiliations:

  • Institute of Formal and applied Linguistics;Institute of Formal and applied Linguistics;Institute of Formal and applied Linguistics

  • Venue:
  • COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics: Posters
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present several ways of measuring the inter-annotator agreement in the ongoing annotation of semantic inter-sentential discourse relations in the Prague Dependency Treebank (PDT). Two ways have been employed to overcome limitations of measuring the agreement on the exact location of the start/end points of the relations. Both methods -- skipping one tree level in the start/end nodes, and the connective-based measure - are focused on a recognition of the existence and of the type of the relations, rather than on fixing the exact positions of the start/end points of the connecting arrows.