Findings of the 2009 workshop on statistical machine translation

Authors:
Chris Callison-Burch;Philipp Koehn;Christof Monz;Josh Schroeder
Affiliations:
Johns Hopkins University;University of Edinburgh;University of Amsterdam;University of Edinburgh
Venue:
StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Year:
2009

Citing 37
Cited 71

Fast and Accurate Sentence Alignment of Bilingual Corpora

AMTA '02 Proceedings of the 5th Conference of the Association for Machine Translation in the Americas on Machine Translation: From Research to Real Users
BLEU: a method for automatic evaluation of machine translation

ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics

ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
Word lattices for multi-source translation

EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
(Meta-) evaluation of machine translation

StatMT '07 Proceedings of the Second Workshop on Statistical Machine Translation
Linguistic features for automatic evaluation of heterogenous MT systems

StatMT '07 Proceedings of the Second Workshop on Statistical Machine Translation
Further meta-evaluation of machine translation

StatMT '08 Proceedings of the Third Workshop on Statistical Machine Translation
METEOR, M-BLEU and M-TER: evaluation metrics for high-correlation with human rankings of machine translation output

StatMT '08 Proceedings of the Third Workshop on Statistical Machine Translation
A smorgasbord of features for automatic MT evaluation

StatMT '08 Proceedings of the Third Workshop on Statistical Machine Translation
Findings of the 2009 workshop on statistical machine translation

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Syntax-oriented evaluation measures for machine translation output

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Combining multi-engine translations with Moses

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
CMU system combination for WMT'09

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
The RWTH system combination system for WMT 2009

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Machine translation system combination with flexible word ordering

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Incremental hypothesis alignment with flexible matching for building confusion networks: BBN system description for WMT09 system combination task

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
The RWTH machine translation system for WMT 2009

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Translation combination using factored word substitution

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
NUS at WMT09: domain adaptation experiments for English-Spanish machine translation of news commentary text

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
The Universität Karlsruhe translation system for the EACL-WMT 2009

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
The TALP-UPC phrase-based translation system for EACL-WMT 2009

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Deep linguistic multilingual translation and bilingual dictionaries

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
MaTrEx: the DCU MT system for WMT 2009

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
LIMSI's statistical translation systems for WMT'09

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
NICT@WMT09: model adaptation and transliteration for Spanish-English SMT

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Statistical post editing and dictionary extraction: Systran/Edinburgh submissions for ACL-WMT2009

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Experiments in morphosyntactic processing for translating to and from German

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Improving alignment for SMT by reordering and augmenting the training corpus

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
English-Czech MT in 2008

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
SMT and SPE machine translation systems for WMT'09

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Joshua: an open source toolkit for parsing-based machine translation

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
An improved statistical transfer system for French--English machine translation

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
The University of Maryland statistical machine translation system for the Fourth Workshop on Machine Translation

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Toward using morphology in French-English phrase-based SMT

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Edinburgh's submission to all tracks of the WMT2009 shared task with reordering and speed improvements to Moses

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Fluency, adequacy, or HTER?: exploring different human judgments with a tunable MT metric

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Manual and automatic evaluation of machine translation between European languages

StatMT '06 Proceedings of the Workshop on Statistical Machine Translation

Findings of the 2009 workshop on statistical machine translation

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Demonstration of Joshua: an open source toolkit for parsing-based machine translation

ACLDemos '09 Proceedings of the ACL-IJCNLP 2009 Software Demonstrations
Feasibility of human-in-the-loop minimum error rate training

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
Fast, cheap, and creative: evaluating translation quality using Amazon's Mechanical Turk

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
The Meteor metric for automatic evaluation of machine translation

Machine Translation
MaxSim: performance and effects of translation fluency

Machine Translation
Machine translation evaluation versus quality estimation

Machine Translation
Metrics for MT evaluation: evaluating reordering

Machine Translation
Extending the meteor machine translation evaluation metric to the phrase level

HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Enabling monolingual translators: post-editing vs. options

HLT '10 Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Tackling sparse data issue in machine translation evaluation

ACLShort '10 Proceedings of the ACL 2010 Conference Short Papers
SMT experiments for Romanian and German using JRC-ACQUIS

MRTECEEL '09 Proceedings of the Workshop on Multilingual Resources, Technologies and Evaluation for Central and Eastern European Languages
Improving SMT for Baltic Languages with Factored Models

Proceedings of the 2010 conference on Human Language Technologies -- The Baltic Perspective: Proceedings of the Fourth International Conference Baltic HLT 2010
Using Mechanical Turk to build machine translation evaluation sets

CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
The RALI machine translation system for WMT 2010

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
Exodus: exploring SMT for EU institutions

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
LIUM SMT machine translation system for WMT 2010

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
Reproducible results in parsing-based machine translation: the JHU shared task submission

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
Jane: open source hierarchical translation, extended with reordering and lexicon models

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
LRscore for evaluating lexical and reordering quality in MT

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
Document-level automatic MT evaluation based on discourse representations

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
TESLA: translation evaluation of sentences with linear-programming-based analysis

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
N-best reranking by multitask learning

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
A hybrid morpheme-word representation for machine translation of morphologically rich languages

EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
"Poetic" statistical machine translation: rhyme and meter

EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
PEM: a paraphrase evaluation metric exploiting parallel texts

EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
Assessing phrase-based translation models with oracle decoding

EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
Fluency constraints for minimum Bayes-risk decoding of statistical machine translation lattices

COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics
Automatically learning source-side reordering rules for large scale machine translation

COLING '10 Proceedings of the 23rd International Conference on Computational Linguistics
Monte Carlo techniques for phrase-based translation

Machine Translation
Linguistic measures for automatic machine translation evaluation

Machine Translation
Pre- and postprocessing for statistical machine translation into Germanic languages

HLT-SS '11 Proceedings of the ACL 2011 Student Session
Reordering metrics for MT

HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Identifying parallel documents from a large bilingual collection of texts: application to parallel article extraction in Wikipedia

BUCC '11 Proceedings of the 4th Workshop on Building and Using Comparable Corpora: Comparable Corpora and the Web
Automatic projection of semantic structures: an application to pairwise translation ranking

SSST-5 Proceedings of the Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation
An evaluation and possible improvement path for current SMT behavior on ambiguous nouns

SSST-5 Proceedings of the Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation
Automatic translation error analysis

TSD'11 Proceedings of the 14th international conference on Text, speech and dialogue
Towards automatic error analysis of machine translation output

Computational Linguistics
Comparing phrase-based and syntax-based paraphrase generation

MTTG '11 Proceedings of the Workshop on Monolingual Text-To-Text Generation
Translating from complex to simplified sentences

PROPOR'10 Proceedings of the 9th international conference on Computational Processing of the Portuguese Language
Findings of the 2011 Workshop on Statistical Machine Translation

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Evaluate with confidence estimation: machine ranking of translation outputs using grammatical features

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Approximating a deep-syntactic metric for MT evaluation and tuning

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Evaluation without references: IBM1 scores as evaluation metrics

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Morphemes and POS tags for n-gram based evaluation metrics

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Regression and ranking based optimisation for sentence level machine translation evaluation

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
The UZH system combination system for WMT 2011

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Better evaluation metrics lead to better machine translation

EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Corroborating text evaluation results with heterogeneous measures

EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Data-driven response generation in social media

EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Minimum bayes risk decoding with enlarged hypothesis space in system combination

CICLing'12 Proceedings of the 13th international conference on Computational Linguistics and Intelligent Text Processing - Volume Part II
Learning to translate: a statistical and computational analysis

Advances in Artificial Intelligence
Study and correlation analysis of linguistic, perceptual, and automatic machine translation evaluations

Journal of the American Society for Information Science and Technology
Adaptation of statistical machine translation model for cross-lingual information retrieval in a service context

EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
Does more data always yield better translations?

EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
ONTS: "optima" news translation system

EACL '12 Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics
Expectations of word sense in parallel corpora

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Character-level machine translation evaluation for languages with ambiguous word boundaries

ACL '12 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1
Probabilistic finite state machines for regression-based MT evaluation

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
Multilingual sentiment analysis using machine translation?

WASSA '12 Proceedings of the 3rd Workshop in Computational Approaches to Subjectivity and Sentiment Analysis
Findings of the 2012 workshop on statistical machine translation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Class error rates for evaluation of machine translation output

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
SPEDE: probabilistic edit distance metrics for MT evaluation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
The CMU-avenue French-English translation system

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Analysing the effect of out-of-domain data on SMT systems

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation
Oracle decoding as a new way to analyze phrase-based machine translation

Machine Translation
Comparative experiments using supervised learning and machine translation for multilingual sentiment analysis

Computer Speech and Language
Fusion of word and letter based metrics for automatic MT evaluation

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Sentence-level ranking with quality estimation

Machine Translation
An intelligent Web agent that autonomously learns how to translate

Web Intelligence and Agent Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents the results of the WMT09 shared tasks, which included a translation task, a system combination task, and an evaluation task. We conducted a large-scale manual evaluation of 87 machine translation systems and 22 system combination entries. We used the ranking of these systems to measure how strongly automatic metrics correlate with human judgments of translation quality, for more than 20 metrics. We present a new evaluation technique whereby system output is edited and judged for correctness.