Mutual information approximation via maximum likelihood estimation of density ratio

Authors:
Taiji Suzuki;Masashi Sugiyama;Toshiyuki Tanaka
Affiliations:
Dept. of Mathematical Informatics, The University of Tokyo;Dept. of Computer Science, Tokyo Institute of Technology;Dept. of Systems Science, Kyoto University
Venue:
ISIT'09 Proceedings of the 2009 IEEE international conference on Symposium on Information Theory - Volume 1
Year:
2009

Citing 3
Cited 3

Edgeworth Approximation of Multivariate Differential Entropy

Neural Computation
Estimation of the information by an adaptive partitioning of the observation space

IEEE Transactions on Information Theory
Divergence Estimation of Continuous Distributions Based on Data-Dependent Partitions

IEEE Transactions on Information Theory

Density Ratio Estimation: A New Versatile Tool for Machine Learning

ACML '09 Proceedings of the 1st Asian Conference on Machine Learning: Advances in Machine Learning
Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search

Neural Networks
Computational complexity of kernel-based density-ratio estimation: a condition number analysis

Machine Learning

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose a new method of approximating mutual information based on maximurn likelihood estimation of a density ratio function. The proposed method, Maximum Likelihood Mutual Information (MLMI), possesses useful properties, e.g., it does not involve density estimation, the global optimal solution can be efficiently computed, it has suitable convergence properties, and model selection criteria are available. Numerical experiments show that MLMI compares favorably with existing methods.