The bootstrapping of the Yarowsky algorithm in real corpora

  • Authors:
  • Ricardo Sánchez-de-Madariaga;José R. Fernández-del-Castillo

  • Affiliations:
  • Departamento de Ciencias de la Computación, Escuela Técnica Superior de Ingenierıa Informática, Politécnico, Campus Universitario, Universidad de Alcalá, 28871 Alcal& ...;Departamento de Ciencias de la Computación, Escuela Técnica Superior de Ingenierıa Informática, Politécnico, Campus Universitario, Universidad de Alcalá, 28871 Alcal& ...

  • Venue:
  • Information Processing and Management: an International Journal
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

The Yarowsky bootstrapping algorithm resolves the homograph-level word sense disambiguation (WSD) problem, which is the sense granularity level required for real natural language processing (NLP) applications. At the same time it resolves the knowledge acquisition bottleneck problem affecting most WSD algorithms and can be easily applied to foreign language corpora. However, this paper shows that the Yarowsky algorithm is significantly less accurate when applied to domain fluctuating, real corpora. This paper also introduces a new bootstrapping methodology that performs much better when applied to these corpora. The accuracy achieved in non-domain fluctuating corpora is not reached due to inherent domain fluctuation ambiguities.