Combining forward and backward WTA for partially activated neural networks

Authors:
Ryotaro Kamimura
Affiliations:
Tokai University, Hiratsuka Kanagawa, Japan
Venue:
ISC '07 Proceedings of the 10th IASTED International Conference on Intelligent Systems and Control
Year:
2007

Citing 5
Cited 0

Competitive learning algorithms for vector quantization

Neural Networks
The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals

Neural Computation
Self-organizing maps

Self-organizing maps
Information-Theoretic Competitive Learning with Inverse Euclidean Distance Output Units

Neural Processing Letters
Partially activated neural networks by controlling information

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we introduce a new type of learning procedure in a partially activated network. The partially activated network has been introduced to simplify a complex hierarchical network. In partially activation, only some parts of the network are activated and used to produce outputs. In the previous studies, we used the conventional winner-takes all algorithm to activate a neuron. However, we have found that the conventional WTA has been ineffective for certain problems. Thus, we introduce a new type of competition, that is, backward winner-takes-all. In this method, a winner is chosen by seeing the error back-propagated from the outputs. We apply this type of partially activated network to a student survey. Experimental results showed that the previous method with conventional WTA could not produce appropriate outputs. On the other hand, the new method could immediately solve the problem. These results show that partially activated networks with backward WTA can be very effective in learning for some problems that conventional WTA could not solve.