Towards predicting post-editing productivity

Authors:
Sharon O'Brien
Affiliations:
School of Applied Language and Intercultural Studies, Centre for Translation and Textual Studies, Centre for Next Generation Localisation, Dublin City University, Dublin, Ireland
Venue:
Machine Translation
Year:
2011

Citing 10
Cited 0

Eye Tracking Methodology: Theory and Practice

Eye Tracking Methodology: Theory and Practice
BLEU: a method for automatic evaluation of machine translation

ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Confidence estimation for machine translation

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Automatic evaluation of machine translation quality using n-gram co-occurrence statistics

HLT '02 Proceedings of the second international conference on Human Language Technology Research
Further meta-evaluation of machine translation

StatMT '08 Proceedings of the Third Workshop on Statistical Machine Translation
Improving word alignment with language model based confidence scores

StatMT '08 Proceedings of the Third Workshop on Statistical Machine Translation
MaTrEx: the DCU MT system for WMT 2009

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Introduction to the special issue on "Automated Metrics for Machine Translation Evaluation"

Machine Translation
Machine translation evaluation versus quality estimation

Machine Translation
Enabling monolingual translators: post-editing vs. options

HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

Quantified Score

Hi-index	0.00

Visualization

Abstract

Machine translation (MT) quality is generally measured via automatic metrics, producing scores that have no meaning for translators who are required to post-edit MT output or for project managers who have to plan and budget for translation projects. This paper investigates correlations between two such automatic metrics (general text matcher and translation edit rate) and post-editing productivity. For the purposes of this paper, productivity is measured via processing speed and cognitive measures of effort using eye tracking as a tool. Processing speed, average fixation time and count are found to correlate well with the scores for groups of segments. Segments with high GTM and TER scores require substantially less time and cognitive effort than medium or low-scoring segments. Future research involving score thresholds and confidence estimation is suggested.