Evaluate with confidence estimation: machine ranking of translation outputs using grammatical features

  • Authors:
  • Eleftherios Avramidis;Maja Popovic;David Vilar;Aljoscha Burchardt

  • Affiliations:
  • German Research Center for Artificial Intelligence (DFKI), Language Technology (LT), Berlin, Germany;German Research Center for Artificial Intelligence (DFKI), Language Technology (LT), Berlin, Germany;German Research Center for Artificial Intelligence (DFKI), Language Technology (LT), Berlin, Germany;German Research Center for Artificial Intelligence (DFKI), Language Technology (LT), Berlin, Germany

  • Venue:
  • WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

We present a pilot study on an evaluation method which is able to rank translation outputs with no reference translation, given only their source sentence. The system employs a statistical classifier trained upon existing human rankings, using several features derived from analysis of both the source and the target sentences. Development experiments on one language pair showed that the method has considerably good correlation with human ranking when using features obtained from a PCFG parser.