Diversity-aware evaluation for paraphrase patterns

  • Authors:
  • Hideki Shima;Teruko Mitamura

  • Affiliations:
  • Carnegie Mellon University, Pittsburgh, PA;Carnegie Mellon University, Pittsburgh, PA

  • Venue:
  • TIWTE '11 Proceedings of the TextInfer 2011 Workshop on Textual Entailment
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

Common evaluation metrics for paraphrase patterns do not necessarily correlate with extrinsic recognition task performance. We propose a metric which gives weight to lexical variety in paraphrase patterns; our proposed metric has a positive correlation with paraphrase recognition task performance, with a Pearson correlation of 0.5~0.7 (k=10, with "strict" judgment) in a statistically significant level (p-value