An information-theoretic approach to feature extraction in competitive learning

Authors:
Ryotaro Kamimura;Tadanari Taniguchi;Ryozo Kitajima
Affiliations:
Tokai University, Hiratsuka, Kanagawa, Japan;Tokai University, Hiratsuka, Kanagawa, Japan;Tokai University, Hiratsuka, Kanagawa, Japan
Venue:
AIA '08 Proceedings of the 26th IASTED International Conference on Artificial Intelligence and Applications
Year:
2008

Citing 4
Cited 0

Competitive learning algorithms for vector quantization

Neural Networks
Feature discovery by competitive learning

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals

Neural Computation
Information-Theoretic Competitive Learning with Inverse Euclidean Distance Output Units

Neural Processing Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we propose a new information-theoretic approach to competitive learning and self-organizing maps. We use several information-theoretic measures such as conditional information and information losses to extract main features in input patterns. For each competitive unit, conditional information content is used to show how much information on input patterns is contained. In addition, for detecting the importance of each variable, information losses are introduced. The information loss is defined by difference between information with all input units and information without an input unit. We applied the method to an artificial data, the Iris problem and a student survey. In all cases, experimental results showed that main features in input patterns were clearly detected.