Combining naive bayes and n-gram language models for text classification

  • Authors:
  • Fuchun Peng;Dale Schuurmans

  • Affiliations:
  • School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada;School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada

  • Venue:
  • ECIR'03 Proceedings of the 25th European conference on IR research
  • Year:
  • 2003

Quantified Score

Hi-index 0.00

Visualization

Abstract

We augment the naive Bayes model with an n-gram language model to address two shortcomings of naive Bayes text classifiers. The chain augmented naive Bayes classifiers we propose have two advantages over standard naive Bayes classifiers. First, a chain augmented naive Bayes model relaxes some of the independence assumptions of naive Bayes--allowing a local Markov chain dependence in the observed variables--while still permitting efficient inference and learning. Second, smoothing techniques from statistical language modeling can be used to recover better estimates than the Laplace smoothing techniques usually used in naive Bayes classification. Our experimental results on three real world data sets show that we achieve substantial improvements over standard naive Bayes classification, while also achieving state of the art performance that competes with the best known methods in these cases.