Backpropagation applied to handwritten zip code recognition
Neural Computation
Methods to speed up error back-propagation learning algorithm
ACM Computing Surveys (CSUR)
Advances in Feedforward Neural Networks: Demystifying Knowledge Acquiring Black Boxes
IEEE Transactions on Knowledge and Data Engineering
| Hi-index | 0.00 |
A rule for the selection of the learning coefficient, @h, for use in back propagation with batch training of neural networks is presented. The length of the error gradient is shown to increase as more training set examples are presented. This results in slow training or nonconvergence if @h is not decreased as the number of input examples increases. The effect of a momentum term is shown to allow a range of @h's to produce similar training rates. Two networks having identical topology are trained at different tasks, one with few training patterns (16) and one with many (192). Distinctly different values of @h are shown to produce good training for the two networks. We propose selecting @h equal to 1.5 divided by the square root of the sum of the squares of the number of each input pattern type. Any group of similar inputs that map to identical outputs constitutes a pattern type. This rule produces a fixed value of @h that yields rapid training when coupled with a momentum coefficient of 0.9 for a wide variety of networks.