Monotonic convergence in an information-theoretic law of small numbers

Authors:
Yaming Yu
Affiliations:
Department of Statistics, University of California, Irvine, CA
Venue:
IEEE Transactions on Information Theory
Year:
2009

Citing 10
Cited 3

Ultra logconcave sequences and negative dependence

Journal of Combinatorial Theory Series A
Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing)

Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing)
On the entropy of compound distributions on nonnegative integers

IEEE Transactions on Information Theory
Concavity of entropy under thinning

ISIT'09 Proceedings of the 2009 IEEE international conference on Symposium on Information Theory - Volume 1
Binomial and Poisson distributions as maximum entropy distributions

IEEE Transactions on Information Theory
Rate of convergence to Poisson law in terms of information divergence

IEEE Transactions on Information Theory
Entropy and the law of small numbers

IEEE Transactions on Information Theory
Monotonic Decrease of the Non-Gaussianness of the Sum of Independent Random Variables: A Simple Proof

IEEE Transactions on Information Theory
Generalized Entropy Power Inequalities and Monotonicity Properties of Information

IEEE Transactions on Information Theory
On the Maximum Entropy Properties of the Binomial Distribution

IEEE Transactions on Information Theory

Sharp bounds on the entropy of the poisson law and related quantities

IEEE Transactions on Information Theory
Thinning, entropy, and the law of thin numbers

IEEE Transactions on Information Theory
Monotonicity, thinning, and discrete versions of the entropy power inequality

IEEE Transactions on Information Theory

Quantified Score

Hi-index	755.02

Visualization

Abstract

An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein et al. 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic convergence in relative entropy is established for general discrete distributions, while monotonic increase of Shannon entropy is proved for the special class of ultra-log-concave distributions. Overall we extend the parallel between the information-theoretic central limit theorem and law of small numbers explored by Kontoyiannis et al. (2005) and Harremoës et al. (2007, 2008, 2009). Ingredients in the proofs include convexity, majorization, and stochastic orders.