Elements of information theory
Elements of information theory
Information theory and statistics: a tutorial
Communications and Information Theory
Monotonic convergence in an information-theoretic law of small numbers
IEEE Transactions on Information Theory
Sharp bounds on the entropy of the poisson law and related quantities
IEEE Transactions on Information Theory
Binomial and Poisson distributions as maximum entropy distributions
IEEE Transactions on Information Theory
Refinements of Pinsker's inequality
IEEE Transactions on Information Theory
Rate of convergence to Poisson law in terms of information divergence
IEEE Transactions on Information Theory
Entropy and the law of small numbers
IEEE Transactions on Information Theory
Monotonicity, thinning, and discrete versions of the entropy power inequality
IEEE Transactions on Information Theory
| Hi-index | 754.90 |
Rényi's thinning operation on a discrete random variable is a natural discrete analog of the scaling operation for continuous random variables. The properties of thinning are investigated in an information-theoretic context, especially in connection with information-theoretic inequalities related to Poisson approximation results. The classical Binomial-to-Poisson convergence (sometimes referred to as the "law of small numbers") is seen to be a special case of a thinning limit theorem for convolutions of discrete distributions. A rate of convergence is provided for this limit, and nonasymptotic bounds are also established. This development parallels, in part, the development of Gaussian inequalities leading to the information-theoretic version of the central limit theorem. In particular, a "thinning Markov chain" is introduced, and it is shown to play a role analogous to that of the Ornstein-Uhlenbeck process in connection to the entropy power inequality.