A Probabilistic Upper Bound on Differential Entropy

Authors:
E. Learned-Miller;J. DeStefano
Affiliations:
Dept. of Comput. Sci., Univ. of Massachusetts, Amherst, MA;-
Venue:
IEEE Transactions on Information Theory
Year:
2008

Citing 0
Cited 1

A finite-sample, distribution-free, probabilistic lower bound on mutual information

Neural Computation

Quantified Score

Hi-index	754.84

Visualization

Abstract

A novel probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the unknown distribution is required. Previous distribution-free bounds on the cumulative distribution function of a random variable given a sample of that variable are used to construct the bound. A simple, fast, and intuitive algorithm for computing the entropy bound from a sample is provided.