Continuous space translation models with neural networks

  • Authors:
  • Le Hai Son;Alexandre Allauzen;François Yvon

  • Affiliations:
  • Univ. Paris-Sud, France and LIMSI/CNRS, Orsay cedex, France;Univ. Paris-Sud, France and LIMSI/CNRS, Orsay cedex, France;Univ. Paris-Sud, France and LIMSI/CNRS, Orsay cedex, France

  • Venue:
  • NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

The use of conventional maximum likelihood estimates hinders the performance of existing phrase-based translation models. For lack of sufficient training data, most models only consider a small amount of context. As a partial remedy, we explore here several continuous space translation models, where translation probabilities are estimated using a continuous representation of translation units in lieu of standard discrete representations. In order to handle a large set of translation units, these representations and the associated estimates are jointly computed using a multi-layer neural network with a SOUL architecture. In small scale and large scale English to French experiments, we show that the resulting models can effectively be trained and used on top of a n-gram translation system, delivering significant improvements in performance.