Mutual Information Estimation in Higher Dimensions: A Speed-Up of a k-Nearest Neighbor Based Estimator

  • Authors:
  • Martin Vejmelka;Kateřina Hlaváčková-Schindler

  • Affiliations:
  • Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod Vodárenskou Věží 2, 18207 Praha 8, Czech Republic;Commission for Scientific Visualization, Austrian Academy of Sciences, Donau-City Str. 1, A-1220 Vienna, Austria

  • Venue:
  • ICANNGA '07 Proceedings of the 8th international conference on Adaptive and Natural Computing Algorithms, Part I
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

We focus on the recently introduced nearest neighbor based entropy estimator from Kraskov, Stögbauer and Grassberger (KSG) [10], the nearest neighbor search of which is performed by the so called box assisted algorithm [7]. We compare the performance of KSG with respect to three spatial indexing methods: box-assisted, k-D trie and projection method, on a problem of mutual information estimation of a variety of pdfs and dimensionalities. We conclude that the k-D trie method is significantly faster then box-assisted search in fixed-mass and fixed-radius neighborhood searches in higher dimensions. The projection method is much slower than both alternatives and not recommended for practical use.