Free energy-based competitive learning for self-organizing maps

Authors:
Ryotaro Kamimura
Affiliations:
Tokai University, Kitakaname Hiratsuka Kanagawa, Japan
Venue:
AIA '08 Proceedings of the 26th IASTED International Conference on Artificial Intelligence and Applications
Year:
2008

Citing 4
Cited 8

Competitive learning algorithms for vector quantization

Neural Networks
Feature discovery by competitive learning

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals

Neural Computation
Information-Theoretic Competitive Learning with Inverse Euclidean Distance Output Units

Neural Processing Letters

An information-theoretic approach to feature extraction in competitive learning

Neurocomputing
Self-enhancement learning: self-supervised and target-creating learning

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Selective enhancement learning in competitive learning

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Self-supervised learning by information enhancement: target-generating and spontaneous learning for competitive learning

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Information-theoretic enhancement learning and its application to visualization of self-organizing maps

Neurocomputing
Selective information enhancement learning for creating interpretable representations in competitive learning

Neural Networks
Double enhancement learning for explicit internal representations: unifying self-enhancement and information enhancement to incorporate information on input variables

Applied Intelligence
Controlling Relations between the Individuality and Collectivity of Neurons and its Application to Self-Organizing Maps

Neural Processing Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we propose a new information-theoretic approach to self-organizing maps. We have so far proposed mutual information maximization to realize competitive learning. However, the computational complexity and fidelity to input patterns become serious when we try to apply it to self-organizing maps. To overcome these short-comings, we introduce a free energy similar to that of statistical mechanics. By the free energy, we need not directly compute mutual information to simplify greatly computational procedures. In addition, in the free energy, errors between targets and outputs are naturally built in. This property can solve the problem of fidelity to input patterns of mutual information maximization. In the free energy, we can increase mutual information, taking due attention to errors between targets and connection weights. To demonstrate the performance of our free energy method, we applied the method to the famous Iris Problem. Experimental results showed that feature maps obtained by free energy minimization was significantly similar to those by the conventional SOM.