Collective information-theoretic competitive learning: emergency of improved performance by collectively treated neurons

Authors:
Ryotaro Kamimura;Fumihiko Yoshida;Ryozo Kitajima
Affiliations:
Information Science Laboratory;Department of Media Studies, Tokai University, Hiratsuka Kanagawa, Japan;Information Science Laboratory
Venue:
ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
Year:
2006

Citing 7
Cited 0

Elements of information theory

Elements of information theory
An information-maximization approach to blind separation and blind deconvolution

Neural Computation
Information-Theoretic Competitive Learning with Inverse Euclidean Distance Output Units

Neural Processing Letters
Vector quantization using information theoretic concepts

Natural Computing: an international journal
2005 Special Issue: Unifying cost and information in information-theoretic competitive learning

Neural Networks - 2005 Special issue: IJCNN 2005
Local synaptic learning rules suffice to maximize mutual information in a linear network

Neural Computation
How to generate ordered maps by maximizing the mutual information between input and output signals

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we try to show that the simple collection of competitive units can show some emergent property such as improved generalization performance. We have so far defined information-theoretic competitive learning with respect to individual competitive units. As information is increased, one competitive unit tends to win the competition. This means that competitive learning can be described as a process of information maximization. However, in living systems, a large number of neurons behave collectively. Thus, it is urgently needed to introduce collective property in information-theoretic competitive learning. In this context, we try to treat several competitive units as one unit, that is, one collective unit. Then, we try to maximize information content not in individual competitive units but in collective competitive units. We applied the method to an artificial data and cabinet approval rating estimation. In all cases, we successfully demonstrated that improved generalization could be obtained.