Estimating Squared-Loss Mutual Information for Independent Component Analysis

Authors:
Taiji Suzuki;Masashi Sugiyama
Affiliations:
Department of Mathematical Informatics, The University of Tokyo, Tokyo, Japan 113-8656;Department of Computer Science, Tokyo Institute of Technology, Tokyo, Japan 152-8552
Venue:
ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Year:
2009

Citing 6
Cited 2

Blind separation of sources, Part 1: an adaptive algorithm based on neuromimetic architecture

Signal Processing
Independent component analysis, a new concept?

Signal Processing - Special issue on higher order statistics
Kernel independent component analysis

The Journal of Machine Learning Research
Convex Optimization

Convex Optimization
Sequential fixed-point ica based on mutual information minimization

Neural Computation
Fast and robust fixed-point algorithms for independent component analysis

IEEE Transactions on Neural Networks

Density Ratio Estimation: A New Versatile Tool for Machine Learning

ACML '09 Proceedings of the 1st Asian Conference on Machine Learning: Advances in Machine Learning
Direct density-ratio estimation with dimensionality reduction via least-squares hetero-distributional subspace search

Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Accurately evaluating statistical independence among random variables is a key component of Independent Component Analysis (ICA). In this paper, we employ a squared-loss variant of mutual information as an independence measure and give its estimation method. Our basic idea is to estimate the ratio of probability densities directly without going through density estimation, by which a hard task of density estimation can be avoided. In this density-ratio approach, a natural cross-validation procedure is available for model selection. Thanks to this, all tuning parameters such as the kernel width or the regularization parameter can be objectively optimized. This is a highly useful property in unsupervised learning problems such as ICA. Based on this novel independence measure, we develop a new ICA algorithm named Least-squares Independent Component Analysis (LICA).