From N-grams to collocations: an evaluation of Xtract

  • Authors:
  • Frank A. Smadja

  • Affiliations:
  • Columbia University, New York, NY

  • Venue:
  • ACL '91 Proceedings of the 29th annual meeting on Association for Computational Linguistics
  • Year:
  • 1991

Quantified Score

Hi-index 0.02

Visualization

Abstract

In previous papers we presented methods for retrieving collocations from large samples of texts. We described a tool, Xtract, that implements these methods and able to retrieve a wide range of collocations in a two stage process. These methods as well as other related methods however have some limitations. Mainly, the produced collocations do not include any kind of functional information and many of them are invalid. In this paper we introduce methods that address these issues. These methods are implemented in an added third stage to Xtract that examines the set of collocations retrieved during the previous two stages to both filter out a number of invalid collocations and add useful syntactic information to the retained ones. By combining parsing and statistical techniques the addition of this third stage has raised the overall precision level of Xtract from 40% to 80% with a precision of 94%. In the paper we describe the methods and the evaluation experiments.