Findings of the 2012 workshop on statistical machine translation

Authors:
Chris Callison-Burch;Philipp Koehn;Christof Monz;Matt Post;Radu Soricut;Lucia Specia
Affiliations:
Johns Hopkins University;University of Edinburgh;University of Amsterdam;Johns Hopkins University;SDL Language Weaver;University of Sheffield
Venue:
WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Year:
2012

Citing 53
Cited 19

(Meta-) evaluation of machine translation

StatMT '07 Proceedings of the Second Workshop on Statistical Machine Translation
Further meta-evaluation of machine translation

StatMT '08 Proceedings of the Third Workshop on Statistical Machine Translation
Findings of the 2009 workshop on statistical machine translation

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Deep linguistic multilingual translation and bilingual dictionaries

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Manual and automatic evaluation of machine translation between European languages

StatMT '06 Proceedings of the Workshop on Statistical Machine Translation
Fast, cheap, and creative: evaluating translation quality using Amazon's Mechanical Turk

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
LIBSVM: A library for support vector machines

ACM Transactions on Intelligent Systems and Technology (TIST)
"This sentence is wrong." Detecting errors in machine-translated sentences

Machine Translation
A grain of salt for the WMT manual evaluation

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Findings of the 2011 Workshop on Statistical Machine Translation

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Meteor 1.3: automatic metric for reliable optimization and evaluation of machine translation systems

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Approximating a deep-syntactic metric for MT evaluation and tuning

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Regression and ranking based optimisation for sentence level machine translation evaluation

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Putting human assessments of machine translation systems in order

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Semantic textual similarity for MT evaluation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Improving AMBER, an MT evaluation metric

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
TerrorCat: a translation error categorization-based MT quality metric

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Class error rates for evaluation of machine translation output

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
SPEDE: probabilistic edit distance metrics for MT evaluation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Quality estimation for machine translation output using linguistic analysis and decoding features

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Black box features for the WMT 2012 quality estimation shared task

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Linguistic features for quality estimation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
PRHLT submission to the WMT12 quality estimation task

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Tree kernels for machine translation quality estimation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
LORIA system for the WMT12 quality estimation shared task

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Quality estimation: an experimental study using unsupervised similarity measures

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The UPC submission to the WMT 2012 shared task on quality estimation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
DCU-symantec submission for the WMT 2012 quality estimation task

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The SDL language weaver systems in the WMT12 quality estimation shared task

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Regression with phrase indicators for estimating MT quality

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Probes in a taxonomy of factored phrase-based models

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The CMU-avenue French-English translation system

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Formemes in English-Czech deep syntactic MT

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The TALP-UPC phrase-based translation systems for WMT12: morphology simplification and domain adaptation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Joshua 4.0: packing, PRO, and paraphrases

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Syntax-aware phrase-based statistical machine translation: system description

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
QCRI at WMT12: experiments in Spanish-English and German-English machine translation of news text

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The RWTH Aachen machine translation system for WMT 2012

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Machine learning for hybrid machine translation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Towards effective use of training data in statistical machine translation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Joint WMT 2012 submission of the QUAERO project

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
LIMSI @ WMT'12

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
UPM system for WMT 2012

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
PROMT DeepHybrid system for WMT12 shared translation task

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The Karlsruhe institute of technology translation systems for the WMT 2012

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Kriya - The SFU system for translation task at WMT-12

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
DEPFIX: a system for automatic correction of Czech MT outputs

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
LIUM's SMT machine translation systems for WMT 2012

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Selecting data for English-to-Czech machine translation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
DFKI's SMT system for WMT 2012

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
GHKM rule extraction and scope-3 parsing in Moses

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Data issues of the multilingual translation matrix

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation

Probabilistic finite state machines for regression-based MT evaluation

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Towards a predicate-argument evaluation for MT

SSST-6 '12 Proceedings of the Sixth Workshop on Syntax, Semantics and Structure in Statistical Translation
Quality estimation for machine translation output using linguistic analysis and decoding features

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Tree kernels for machine translation quality estimation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Quality estimation: an experimental study using unsupervised similarity measures

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The UPC submission to the WMT 2012 shared task on quality estimation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The SDL language weaver systems in the WMT12 quality estimation shared task

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Combining quality prediction and system selection for improved automatic translation output

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Evaluating indirect strategies for Chinese-Spanish statistical machine translation

Journal of Artificial Intelligence Research
The efficacy of human post-editing for language translation

Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Identifying useful human correction feedback from an on-line machine translation service

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Evaluating indirect strategies for Chinese-Spanish statistical machine translation: extended abstract

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Quality estimation for machine translation: some lessons learned

Machine Translation
Sentence-level ranking with quality estimation

Machine Translation
Dimensionality reduction methods for machine translation quality estimation

Machine Translation
Predicting sentence translation quality using extrinsic and language independent features

Machine Translation
Investigating the contribution of linguistic information to quality estimation

Machine Translation
Quality estimation for machine translation: preface

Machine Translation
Weakly supervised approaches for quality estimation

Machine Translation

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents the results of the WMT12 shared tasks, which included a translation task, a task for machine translation evaluation metrics, and a task for run-time estimation of machine translation quality. We conducted a large-scale manual evaluation of 103 machine translation systems submitted by 34 teams. We used the ranking of these systems to measure how strongly automatic metrics correlate with human judgments of translation quality for 12 evaluation metrics. We introduced a new quality estimation task this year, and evaluated submissions from 11 teams.