Measuring statistical dependence via the mutual information dimension

Authors:
Mahito Sugiyama;Karsten M. Borgwardt
Affiliations:
Machine Learning and Computational Biology Research Group, Max Planck Institute for Intelligent Systems and Max Planck Institute for Developmental Biology, Tübingen, Germany;Machine Learning and Computational Biology Research Group, Max Planck Institute for Intelligent Systems and Max Planck Institute for Developmental Biology, Tübingen, Germany and Zentrum f ...
Venue:
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Year:
2013

Citing 6
Cited 0

Mutual information and global strange attractors in Taylor-Couette flow

Physica D
Generalized redundancies for time series analysis

Physica D
Kernel independent component analysis

The Journal of Machine Learning Research
Rényi information dimension: fundamental limits of almost lossless analog compression

IEEE Transactions on Information Theory
Measuring statistical dependence with hilbert-schmidt norms

ALT'05 Proceedings of the 16th international conference on Algorithmic Learning Theory
MMSE Dimension

IEEE Transactions on Information Theory

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose to measure statistical dependence between two random variables by the mutual information dimension (MID), and present a scalable parameter-free estimation method for this task. Supported by sound dimension theory, our method gives an effective solution to the problem of detecting interesting relationships of variables in massive data, which is nowadays a heavily studied topic in many scientific disciplines. Different from classical Pearson's correlation coefficient, MID is zero if and only if two random variables are statistically independent and is translation and scaling invariant. We experimentally show superior performance of MID in detecting various types of relationships in the presence of noise data. Moreover, we illustrate that MID can be effectively used for feature selection in regression.