ParaMetric: an automatic evaluation metric for paraphrasing

  • Authors:
  • Chris Callison-Burch;Trevor Cohn;Mirella Lapata

  • Affiliations:
  • Johns Hopkins University, Baltimore, MD;University of Edinburgh, Edinburgh;University of Edinburgh, Edinburgh

  • Venue:
  • COLING '08 Proceedings of the 22nd International Conference on Computational Linguistics - Volume 1
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present ParaMetric, an automatic evaluation metric for data-driven approaches to paraphrasing. ParaMetric provides an objective measure of quality using a collection of multiple translations whose paraphrases have been manually annotated. ParaMetric calculates precision and recall scores by comparing the paraphrases discovered by automatic paraphrasing techniques against gold standard alignments of words and phrases within equivalent sentences. We report scores for several established paraphrasing techniques.