Self-configurable fault monitoring in ad-hoc networks
Ad Hoc Networks
On the entropy of a hidden Markov process
Theoretical Computer Science
Bayesian inference for nonnegative matrix factorisation models
Computational Intelligence and Neuroscience
Minimum expected length of fixed-to-variable lossless compression of memoryless sources
ISIT'09 Proceedings of the 2009 IEEE international conference on Symposium on Information Theory - Volume 1
Sharp bounds on the entropy of the poisson law and related quantities
IEEE Transactions on Information Theory
Noisy constrained capacity for BSC channels
IEEE Transactions on Information Theory
Fault monitoring in ad-hoc networks based on information theory
NETWORKING'06 Proceedings of the 5th international IFIP-TC6 conference on Networking Technologies, Services, and Protocols; Performance of Computer and Communication Networks; Mobile and Wireless Communications Systems
Discrete Applied Mathematics
| Hi-index | 754.96 |
We investigate the basic question of information theory, namely, evaluation of Shannon entropy, and a more general Renyi (1961) entropy, for some discrete distributions (e.g., binomial, negative binomial, etc.). We aim at establishing analytic methods (i.e., those in which complex analysis plays a pivotal role) for such computations which often yield estimates of unparalleled precision. The main analytic tool used here is that of analytic poissonization and depoissonization. We illustrate our approach on the entropy evaluation of the binomial distribution, that is, we prove that for binomial (n, p) distribution Shannon's hn becomes hn≈½ln n+½+ln√(2πp(1-p))+Σk⩾1ak n-k where ak are explicitly computable constants. Moreover, we argue that analytic methods (e.g., complex asymptotics such as Rice's method and singularity analysis, Mellin transforms, poissonization, and depoissonization) can offer new tools for information theory, especially for studying second-order asymptotics (e.g., redundancy). In fact, there has been a resurgence of interest and a few successful applications of analytic methods to a variety of problems of information theory, therefore, we propose to name such investigations as analytic information theory