Nonparametric entropy estimation for stationary processes and random fields, with applications to English text

  • Authors:
  • I. Kontoyiannis;P. H. Algoet;Yu. M. Suhov;A. J. Wyner

  • Affiliations:
  • Dept. of Electr. Eng., Stanford Univ., CA;-;-;-

  • Venue:
  • IEEE Transactions on Information Theory
  • Year:
  • 2006

Quantified Score

Hi-index 754.84

Visualization

Abstract

We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesaro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker (1940). We provide examples of their performance on English text, and we generalize our results to countable alphabet processes and to random fields