The infomin criterion: an information theoretic unifying objective function for topographic mappings

Authors:
Yoshitatsu Matsuda;Kazunori Yamaguchi
Affiliations:
Kazunori Yamaguchi Laboratory, Department of General Systems Studies, Graduate School of Arts and Sciences, The University of Tokyo, Meguro-ku, Tokyo, Japan;Kazunori Yamaguchi Laboratory, Department of General Systems Studies, Graduate School of Arts and Sciences, The University of Tokyo, Meguro-ku, Tokyo, Japan
Venue:
ICANN/ICONIP'03 Proceedings of the 2003 joint international conference on Artificial neural networks and neural information processing
Year:
2003

Citing 11
Cited 2

Self-Organization in a Perceptual Network

Computer
Original Contribution: Efficient information transfer and anti-Hebbian neural networks

Neural Networks
Toward a theory of the striate cortex

Neural Computation
A Bayesian analysis of self-organizing maps

Neural Computation
A unifying objective function for topographic mappings

Neural Computation
GTM: the generative topographic mapping

Neural Computation
Constrained optimization for neural map formation: a unifying framework for weight growth and normalization

Neural Computation
Topographic Independent Component Analysis

Neural Computation
How to generate ordered maps by maximizing the mutual information between input and output signals

Neural Computation
Review:

Neural Computation
Fast and robust fixed-point algorithms for independent component analysis

IEEE Transactions on Neural Networks

A fixed-point algorithm of topographic ICA

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
The infomin principle for ICA and topographic mappings

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we propose a new objective function for forming topographic mappings, named the "InfoMin" criterion. This criterion is defined as the average of the information transferred through small neighbor areas over a mapping, and its closed form is derived by use of the Edgeworth expansion. If the second-order statistics (namely, normal correlations among neurons) are not zero, the InfoMin criterion is consistent with the C measure (a unifying objective function for topographic mapping proposed by Goodhill and Sejnowski [1]). In addition, the higher-order correlations are dominant in this criterion only if the second-order ones are negligible. So, it can explain many previous models comprehensively, and is applicable to uncorrelated signals such that ZCA or ICA generates as well. Numerical experiments on natural scenes verify that the InfoMin criterion gives a strong unifying framework for topographic mappings based on information theory.