Semiring frameworks and algorithms for shortest-distance problems
Journal of Automata, Languages and Combinatorics
BLEU: a method for automatic evaluation of machine translation
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
The string edit distance matching problem with moves
ACM Transactions on Algorithms (TALG)
ORANGE: a method for evaluating automatic evaluation metrics for machine translation
COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Machine translation system combination using ITG-based alignments
HLT-Short '08 Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers
Fluency, adequacy, or HTER?: exploring different human judgments with a tunable MT metric
StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Novel reordering approaches in phrase-based statistical machine translation
ParaText '05 Proceedings of the ACL Workshop on Building and Using Parallel Texts
The Meteor metric for automatic evaluation of machine translation
Machine Translation
OpenFst: a general and efficient weighted finite-state transducer library
CIAA'07 Proceedings of the 12th international conference on Implementation and application of automata
Handbook of Natural Language Processing and Machine Translation: DARPA Global Autonomous Language Exploitation
A grain of salt for the WMT manual evaluation
WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Findings of the 2011 Workshop on Statistical Machine Translation
WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Meteor 1.3: automatic metric for reliable optimization and evaluation of machine translation systems
WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Hi-index | 0.00 |
It is common knowledge that translation is an ambiguous, 1-to-n mapping process, but to date, our community has produced no empirical estimates of this ambiguity. We have developed an annotation tool that enables us to create representations that compactly encode an exponential number of correct translations for a sentence. Our findings show that naturally occurring sentences have billions of translations. Having access to such large sets of meaning-equivalent translations enables us to develop a new metric, HyTER, for translation accuracy. We show that our metric provides better estimates of machine and human translation accuracy than alternative evaluation metrics.