Partially activated neural networks by controlling information

Authors:
Ryotaro Kamimura
Affiliations:
Information Science Laboratory, Information Technology Center, Tokai University, Kanagawa, Japan
Venue:
ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Year:
2007

Citing 5
Cited 1

Competitive learning algorithms for vector quantization

Neural Networks
Feature discovery by competitive learning

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1
Growing cell structures—a self-organizing network for unsupervised and supervised learning

Neural Networks
The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals

Neural Computation
Information-Theoretic Competitive Learning with Inverse Euclidean Distance Output Units

Neural Processing Letters

Combining forward and backward WTA for partially activated neural networks

ISC '07 Proceedings of the 10th IASTED International Conference on Intelligent Systems and Control

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we propose partial activation to simplify complex neural networks. For choosing important elements in a network, we develop a fully supervised competitive learning that can deal with any targets. This approach is an extension of competitive learning to a more general one, including supervised learning. Because competitive learning focuses on an important competitive unit, all the other competitive units are of no use. Thus, the number of connection weights to be updated can be reduced to a minimum point when we use competitive learning. We apply the method to the XOR problem to show that learning is possible with good interpretability of internal representations. Then, we apply the method to a student survey. In the problem, we try to show that the new method can produce connection weights that are more stable than those produced by BP. In addition, we show that, though connection weights are quite similar to those produced by linear regression analysis, generalization performance can be improved by changing the number of competitive units.