A finite-sample, distribution-free, probabilistic lower bound on mutual information

  • Authors:
  • Nathan D. VanderKraats;Arunava Banerjee

  • Affiliations:
  • -;-

  • Venue:
  • Neural Computation
  • Year:
  • 2011

Quantified Score

Hi-index 0.00

Visualization

Abstract

For any memoryless communication channel with a binary-valued input and a one-dimensional real-valued output, we introduce a probabilistic lower bound on the mutual information given empirical observations on the channel. The bound is built on the Dvoretzky-Kiefer-Wolfowitz inequality and is distribution free. A quadratic time algorithm is described for computing the bound and its corresponding class-conditional distribution functions. We compare our approach to existing techniques and show the superiority of our bound to a method inspired by Fano's inequality where the continuous random variable is discretized.