New statistical methods for phrase break prediction

  • Authors:
  • Helmut Schmid;Michaela Atterer

  • Affiliations:
  • University of Stuttgart, Stuttgart, Germany;University of Stuttgart, Stuttgart, Germany

  • Venue:
  • COLING '04 Proceedings of the 20th international conference on Computational Linguistics
  • Year:
  • 2004

Quantified Score

Hi-index 0.00

Visualization

Abstract

The paper presents two methods for the prediction of phrase breaks. The first method uses a standard HMM part-of-speech tagger with variable context length. The second method directly encodes the distance from the last phrase break in its states. It combines the probability of a phrase break given the distance from the last phrase break with the probability of a break given the local context consisting of the surrounding words and part of speech tags. The accuracy of the new tagger is 2 percentage points higher than that of Taylor and Black (1998) on similar data.