2005 Special Issue: Unifying cost and information in information-theoretic competitive learning

Authors:
Ryotaro Kamimura
Affiliations:
Information Science Laboratory, Tokai University, 1117, Kitakaname Hiratsuka Kanagawa 259-1292, Japan
Venue:
Neural Networks - 2005 Special issue: IJCNN 2005
Year:
2005

Citing 8
Cited 6

Self-Organization in a Perceptual Network

Computer
Competitive learning algorithms for vector quantization

Neural Networks
The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals

Neural Computation
Information-Theoretic Competitive Learning with Inverse Euclidean Distance Output Units

Neural Processing Letters
Vector quantization using information theoretic concepts

Natural Computing: an international journal
Local synaptic learning rules suffice to maximize mutual information in a linear network

Neural Computation
How to generate ordered maps by maximizing the mutual information between input and output signals

Neural Computation
Rival penalized competitive learning for clustering analysis, RBF net, and curve detection

IEEE Transactions on Neural Networks

Feature Discovery by Enhancement and Relaxation of Competitive Units

IDEAL '08 Proceedings of the 9th International Conference on Intelligent Data Engineering and Automated Learning
Enhancing and Relaxing Competitive Units for Feature Discovery

Neural Processing Letters
Structural enhanced information to detect features in competitive learning

SMC'09 Proceedings of the 2009 IEEE international conference on Systems, Man and Cybernetics
Structural enhanced information and its application to improved visualization of self-organizing maps

Applied Intelligence
Automatic inference of cabinet approval ratings by information-theoretic competitive learning

ICONIP'06 Proceedings of the 13th international conference on Neural Information Processing - Volume Part II
Collective information-theoretic competitive learning: emergency of improved performance by collectively treated neurons

ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we introduce costs into the framework of information maximization and try to maximize the ratio of information to its associated cost. We have shown that competitive learning is realized by maximizing mutual information between input patterns and competitive units. One shortcoming of the method is that maximizing information does not necessarily produce representations faithful to input patterns. Information maximizing primarily focuses on some parts of input patterns that are used to distinguish between patterns. Therefore, we introduce the cost, which represents average distance between input patterns and connection weights. By minimizing the cost, final connection weights reflect input patterns well. We applied the method to a political data analysis, a voting attitude problem and a Wisconsin cancer problem. Experimental results confirmed that, when the cost was introduced, representations faithful to input patterns were obtained. In addition, improved generalization performance was obtained within a relatively short learning time.