Bucking the trend: large-scale cost-focused active learning for statistical machine translation

Authors:
Michael Bloodgood;Chris Callison-Burch
Affiliations:
Johns Hopkins University, Baltimore, MD;Speech Processing Johns Hopkins University, Baltimore, MD
Venue:
ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
Year:
2010

Citing 25
Cited 10

A sequential algorithm for training text classifiers

SIGIR '94 Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval
Less is More: Active Learning with Support Vector Machines

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Support vector machine active learning with applications to text classification

The Journal of Machine Learning Research
Scaling to very very large corpora for natural language disambiguation

ACL '01 Proceedings of the 39th Annual Meeting on Association for Computational Linguistics
BLEU: a method for automatic evaluation of machine translation

ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
An empirical study of active learning with support vector machines for Japanese word segmentation

ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Rule writing or annotation: cost-efficient resource usage for base noun phrase chunking

ACL '00 Proceedings of the 38th Annual Meeting on Association for Computational Linguistics
Sample selection for statistical grammar induction

EMNLP '00 Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics - Volume 13
Hierarchical Phrase-Based Translation

Computational Linguistics
A stopping criterion for active learning

Computer Speech and Language
Assessing the costs of sampling methods in active learning for annotation

HLT-Short '08 Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers
An approach to reducing annotation costs for BioNLP

BioNLP '08 Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing
A method for stopping active learning based on stabilizing predictions and the need for user-adjustable stopping

CoNLL '09 Proceedings of the Thirteenth Conference on Computational Natural Language Learning
Cheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks

EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Active learning for statistical phrase-based machine translation

NAACL '09 Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics
Taking into account the differences between actively and passively acquired data: the case of active learning with support vector machines for imbalanced datasets

NAACL-Short '09 Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers
Selective supervision: guiding supervised learning with decision-theoretic active learning

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Joshua: an open source toolkit for parsing-based machine translation

StatMT '09 Proceedings of the Fourth Workshop on Statistical Machine Translation
Syntax augmented machine translation via chart parsing

StatMT '06 Proceedings of the Workshop on Statistical Machine Translation
Active learning for multilingual statistical machine translation

ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1 - Volume 1
Semi-supervised active learning for sequence labeling

ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2
Investigating the effects of selective sampling on the annotation task

CONLL '05 Proceedings of the Ninth Conference on Computational Natural Language Learning
Phrase-based statistical language generation using graphical models and active learning

ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
An active learning approach to finding related terms

ACLShort '10 Proceedings of the ACL 2010 Conference Short Papers
Using Mechanical Turk to build machine translation evaluation sets

CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk

Phrase-based statistical language generation using graphical models and active learning

ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
Creating speech and language data with Amazon's Mechanical Turk

CSLDAMT '10 Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk
Collecting highly parallel data for paraphrase evaluation

HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
An active learning scenario for interactive machine translation

ICMI '11 Proceedings of the 13th international conference on multimodal interfaces
Instance selection for machine translation using feature decay algorithms

WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
Learning to translate: a statistical and computational analysis

Advances in Artificial Intelligence
Active learning for interactive machine translation

EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
Translation model size reduction for hierarchical phrase-based statistical machine translation

ACL '12 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2
Building a personalized audio equalizer interface with transfer learning and active learning

Proceedings of the second international ACM workshop on Music information retrieval with user-centered and multimodal strategies
Implementing crowdsourcing-based relevance experimentation: an industrial perspective

Information Retrieval

Quantified Score

Hi-index	0.00

Visualization

Abstract

We explore how to improve machine translation systems by adding more translation data in situations where we already have substantial resources. The main challenge is how to buck the trend of diminishing returns that is commonly encountered. We present an active learning-style data solicitation algorithm to meet this challenge. We test it, gathering annotations via Amazon Mechanical Turk, and find that we get an order of magnitude increase in performance rates of improvement.