From n-gram-based to CRF-based translation models

  • Authors:
  • Thomas Lavergne;Josep Maria Crego;Alexandre Allauzen;François Yvon

  • Affiliations:
  • LIMSI/CNRS, Orsay, Cédex;LIMSI/CNRS, Orsay, Cédex;LIMSI/CNRS & Uni. Paris Sud, Orsay, Céédex;LIMSI/CNRS & Uni. Paris Sud, Orsay, Céédex

  • Venue:
  • WMT '11 Proceedings of the Sixth Workshop on Statistical Machine Translation
  • Year:
  • 2011

Quantified Score

Hi-index 0.01

Visualization

Abstract

A major weakness of extant statistical machine translation (SMT) systems is their lack of a proper training procedure. Phrase extraction and scoring processes rely on a chain of crude heuristics, a situation judged problematic by many. In this paper, we recast the machine translation problem in the familiar terms of a sequence labeling task, thereby enabling the use of enriched feature sets and exact training and inference procedures. The tractability of the whole enterprise is achieved through an efficient implementation of the conditional random fields (CRFs) model using a weighted finite-state transducers library. This approach is experimentally contrasted with several conventional phrase-based systems.