Self-organizing by information maximization: realizing self-organizing maps by information-theoretic competitive learning

Authors:
Ryotaro Kamimura
Affiliations:
Information Science Laboratory, Information Technology Center, Tokai University, Hiratsuka Kanagawa, Japan
Venue:
ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
Year:
2006

Citing 10
Cited 0

Self-Organization in a Perceptual Network

Computer
Elements of information theory

Elements of information theory
An information-maximization approach to blind separation and blind deconvolution

Neural Computation
The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals

Neural Computation
Self-Organizing Maps

Self-Organizing Maps
Feature extraction by non parametric mutual information maximization

The Journal of Machine Learning Research
Information-Theoretic Competitive Learning with Inverse Euclidean Distance Output Units

Neural Processing Letters
Vector quantization using information theoretic concepts

Natural Computing: an international journal
Local synaptic learning rules suffice to maximize mutual information in a linear network

Neural Computation
How to generate ordered maps by maximizing the mutual information between input and output signals

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

The present paper shows that a self-organizing process can be realized simply by maximizing information between input patterns and competitive units. We have already shown that information maximization corresponds to competitive processes. Thus, if cooperation processes can be incorporated in information maximization, self-organizing maps can naturally be realized by information maximization. By using the weighted sum of distances among neurons or collected distance, we successfully incorporate cooperation processes in the main mechanism of information maximization. For comparing our method with the standard SOM, we applied the method to the well-known artificial data and show that clear feature maps can be obtained by maximizing information.