The Web as a parallel corpus

  • Authors:

  • Philip Resnik;Noah A. Smith

  • Affiliations:

  • Department of Linguistics and Institute for Advanced Computer Studies, University of Maryland, College Park, MD;Department of Computer Science and Center for Language and Speech Processing, John Hopkins University, Baltimore, MD

  • Venue:
  • Computational Linguistics - Special issue on web as corpus
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

Parallel corpora have become an essential resource for work in multilingual natural language processing. In this article, we report on our work using the STRAND system for mining parallel text on the World Wide Web,first reviewing the original algorithm and results and then presenting a set of significant enhancements. These enhancements include the use of supervised learning based on structural features of documents to improve classification performance, a new content-based measure of translational equivalence, and adaptation of the system to take advantage of the Internet Archive for mining parallel text from the Web on a large scale. Finally, the value of these techniques is demonstrated in the construction of a significant parallel corpus for a low-density language pair.