A maximum entropy approach to natural language processing

  • Authors:
  • Adam L. Berger;Vincent J. Della Pietra;Stephen A. Della Pietra

  • Affiliations:
  • Columbia University;Renaissance Technologies;Renaissance Technologies

  • Venue:
  • Computational Linguistics
  • Year:
  • 1996

Quantified Score

Hi-index 0.02

Visualization

Abstract

The concept of maximum entropy can be traced back along multiple threads to Biblical times. Only recently, however, have computers become powerful enough to permit the widescale application of this concept to real world problems in statistical estimation and pattern recognition. In this paper, we describe a method for statistical modeling based on maximum entropy. We present a maximum-likelihood approach for automatically constructing maximum entropy models and describe how to implement this approach efficiently, using as examples several problems in natural language processing.