Work-in-progress project report: CESTA - machine translation evaluation campaign

  • Authors:

  • Widad Mustafa El Hadi;Marianne Dabbadie;Ismaïl Timimi;Martin Rajman;Philippe Langlais;Antony Hartley;Andrei Popescu Belis

  • Affiliations:

  • Université de Lille, Villeneuve d'Ascq, Cedex - France;Université de Lille, Villeneuve d'Ascq, Cedex - France;Université de Lille, Villeneuve d'Ascq, Cedex - France;LIA, Ecole, Polytechnique, Lausanne, Switzerland;Université de Montréal, Montréal (Québec) - Canada;University of Leeds, Leeds, UK;University of Geneva, Geneva, Switzerland

  • Venue:
  • LRTWRT '04 Proceedings of the Second International Workshop on Language Resources for Translation Work, Research and Training
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

CESTA, the first European Campaign dedicated to MT Evaluation, is a project labelled by the French Technolangue action. CESTA provides an evaluation of six commercial and academic MT systems using a protocol set by an international panel of experts. CESTA aims at producing reusable resources and information about reliability of the metrics. Two runs will be carried out: one using the system's basic dictionary, another after terminological adaptation. Evaluation task, test material, resources, evaluation measures, metrics, will be detailed in the full paper. The protocol is the combination of a contrastive reference to: IBM "BLEU" protocol (Papineni, K., S. Roukos, T. Ward and Z. Wei-Jing, 2001); "BLANC" protocol derived from (Hartley, Rajman, 2002).; "ROUGE" protocol (Babych, Hartley, Atwell, 2003). The results of the campaign will be published in a final report and be the object of two intermediary and final workshops.