Universal entropy estimation via block sorting

Authors:
Haixiao Cai;S. R. Kulkarni;S. Verdu
Affiliations:
Electr. Eng. Dept., Princeton Univ., NJ, USA;-;-
Venue:
IEEE Transactions on Information Theory
Year:
2006

Citing 0
Cited 7

Approximating entropy from sublinear samples

SODA '07 Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms
Bias reduction via linear combination of nearest neighbour entropy estimators

International Journal of Information and Coding Theory
Universal Estimation of Information Measures for Analog Sources

Foundations and Trends in Communications and Information Theory
Universal estimation of erasure entropy

IEEE Transactions on Information Theory
The interplay between entropy and variational distance

IEEE Transactions on Information Theory
Estimating the directed information to infer causal relationships in ensemble neural spike train recordings

Journal of Computational Neuroscience
Bounds from a card trick

Journal of Discrete Algorithms

Quantified Score

Hi-index	754.96

Visualization

Abstract

In this correspondence, we present a new universal entropy estimator for stationary ergodic sources, prove almost sure convergence, and establish an upper bound on the convergence rate for finite-alphabet finite memory sources. The algorithm is motivated by data compression using the Burrows-Wheeler block sorting transform (BWT). By exploiting the property that the BWT output sequence is close to a piecewise stationary memoryless source, we can segment the output sequence and estimate probabilities in each segment. Experimental results show that our algorithm outperforms Lempel-Ziv (LZ) string-matching-based algorithms.