BBN TransTalk: Robust multilingual two-way speech-to-speech translation for mobile platforms

Authors:
Rohit Prasad;Prem Natarajan;David Stallard;Shirin Saleem;Shankar Ananthakrishnan;Stavros Tsakalidis;Chia-Lin Kao;Fred Choi;Ralf Meermeier;Mark Rawls;Jacob Devlin;Kriste Krstovski;Aaron Challenner
Affiliations:
Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA;Raytheon BBN Technologies, Speech, Language, and Multimedia Department, 10 Moulton Street, Cambridge, MA, USA
Venue:
Computer Speech and Language
Year:
2013

Citing 7
Cited 2

An Algorithm that Learns What‘s in a Name

Machine Learning - Special issue on natural language learning
A systematic comparison of various statistical alignment models

Computational Linguistics
Statistical phrase-based translation

NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1
Confidence estimation for machine translation

COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Transonics: a practical speech-to-speech translator for English-Farsi medical dialogues

ACLdemo '05 Proceedings of the ACL 2005 on Interactive poster and demonstration sessions
Recent advances in SRI'S IraqComm™ Iraqi Arabic-English speech-to-speech translation system

ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
IBM MASTOR system: multilingual automatic speech-to-speech translator

MST '06 Proceedings of the Workshop on Medical Speech Translation

Talking to machines

Communications of the ACM
Evaluating the Quality of Mobile Medical Speech Translators Based on ISO/IEC 9126 Series: Definition, Weighted Quality Model and Metrics

International Journal of Reliable and Quality E-Healthcare

Quantified Score

Hi-index	0.02

Visualization

Abstract

In this paper we present a speech-to-speech (S2S) translation system called the BBN TransTalk that enables two-way communication between speakers of English and speakers who do not understand or speak English. The BBN TransTalk has been configured for several languages including Iraqi Arabic, Pashto, Dari, Farsi, Malay, Indonesian, and Levantine Arabic. We describe the key components of our system: automatic speech recognition (ASR), machine translation (MT), text-to-speech (TTS), dialog manager, and the user interface (UI). In addition, we present novel techniques for overcoming specific challenges in developing high-performing S2S systems. For ASR, we present techniques for dealing with lack of pronunciation and linguistic resources and effective modeling of ambiguity in pronunciations of words in these languages. For MT, we describe techniques for dealing with data sparsity as well as modeling context. We also present and compare different user confirmation techniques for detecting errors that can cause the dialog to drift or stall.