Training multi-layered neural network neocognitron

• Authors: Kunihiko Fukushima
• Venue: Neural Networks
• Year: 2013

This paper proposes new learning rules suited for training multi-layered neural networks and applies them to the neocognitron. The neocognitron is a hierarchical multi-layered neural network capable of robust visual pattern recognition. It acquires the ability to recognize visual patterns through learning. For training intermediate layers of the hierarchical network of the neocognitron, we use a new learning rule named add-if-silent. By the use of the add-if-silent rule, the learning process becomes much simpler and more stable, and the computational cost for learning is largely reduced. Nevertheless, a high recognition rate can be kept without increasing the scale of the network. For the highest stage of the network, we use the method of interpolating-vector. We have previously reported that the recognition rate is greatly increased if this method is used during recognition. This paper proposes a new method of using it for both learning and recognition. Computer simulation demonstrates that the new neocognitron, which uses the add-if-silent and the interpolating-vector, produces a higher recognition rate for handwritten digits recognition with a smaller scale of the network than the neocognitron of previous versions.