A Winnow-Based Approach to Context-Sensitive Spelling Correction

  • Authors:
  • Andrew R. Golding;Dan Roth

  • Affiliations:
  • MERL—A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MA 02139. golding@merl.com;Department of Computer Science, University of Illinois—Urbana/Champaign, 1304 W. Springfield Avenue, Urbana, IL 61801. danr@cs.uiuc.edu

  • Venue:
  • Machine Learning - Special issue on natural language learning
  • Year:
  • 1999

Quantified Score

Hi-index 0.00

Visualization

Abstract

A large class of machine-learning problems in natural languagerequire the characterization of linguistic context. Two characteristic properties of such problems arethat their feature space is of very high dimensionality,and their target concepts depend on only a small subsetof the features in the space. Under such conditions, multiplicative weight-update algorithmssuch as Winnow have been shown to have exceptionally goodtheoretical properties. In the work reported here, we present an algorithmcombining variants of Winnow and weighted-majority voting,and apply it to a problem in the aforementioned class: context-sensitive spelling correction.This is the task of fixing spelling errors that happen to resultin valid words, such as substituting to for too,casual for causal, and so on.We evaluate our algorithm, WinSpell,by comparing it against BaySpell, a statistics-based methodrepresenting the state of the art for this task.We find: (1) When run with a full (unpruned) set of features,WinSpell achieves accuracies significantly higher than BaySpell was able toachieve in either the pruned or unpruned condition; (2) When compared with other systems in the literature, WinSpell exhibits the highest performance; (3) While several aspects of WinSpell‘s architecturecontribute to its superiority over BaySpell,the primary factor is that it is able to learn a better linear separatorthan BaySpell learns; (4) When run on a test set drawn from a different corpus than the training set was drawn from,WinSpell is better able than BaySpell to adapt,using a strategy we will present that combinessupervised learning on the training setwith unsupervised learning on the (noisy) test set.