Information Maximization and Independent Component Analysis: Is There a Difference?

Authors:
D. Obradovic;G. Deco
Affiliations:
Siemens AG, Central Technology Department, Information and Communications, 81739 Munich, Germany;Siemens AG, Central Technology Department, Information and Communications, 81739 Munich, Germany
Venue:
Neural Computation
Year:
1998

Citing 8
Cited 1

What does the retina know about natural scenes?

Neural Computation
Redundancy reduction as a strategy for unsupervised learning

Neural Computation
Independent component analysis, a new concept?

Signal Processing - Special issue on higher order statistics
Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures

Neural Networks
An information-maximization approach to blind separation and blind deconvolution

Neural Computation
Linear redundancy reduction learning

Neural Networks
Redundancy reduction and independent component analysis: conditions on cumulants and adaptive approaches

Neural Computation
An Information-Theoretic Approach to Neural Computing

An Information-Theoretic Approach to Neural Computing

Sudden-Target search algorithm of monitor area based on visual information processing model

ICIC'12 Proceedings of the 8th international conference on Intelligent Computing Theories and Applications

Quantified Score

Hi-index	0.00

Visualization

Abstract

This article provides a detailed and rigorous analysis of the two commonly used methods for redundancy reduction: linear independent component analysis (ICA) posed as a direct minimization of a suitably chosen redundancy measure and information maximization (InfoMax) of a continuous stochastic signal transmitted through an appropriate nonlinear network. The article shows analytically that ICA based on the KullbackLeibler information as a redundancy measure and InfoMax lead to the same solution if the parameterization of the output nonlinear functions in the latter method is sufficiently rich. Furthermore, this work discusses the alternative redundancy measures not based on the Kullback-Leibler information distance. The practical issues of applying ICA and InfoMax are also discussed and illustrated on the problem of extracting statistically independent factors from a linear, pixel-by-pixel mixture of images.