Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations
Natural gradient works efficiently in learning
Neural Computation
Annealed online learning in multilayer neural networks
On-line learning in neural networks
Information Theory, Inference & Learning Algorithms
Information Theory, Inference & Learning Algorithms
Error Correction Coding: Mathematical Methods and Algorithms
Error Correction Coding: Mathematical Methods and Algorithms
Learning by natural gradient on noncompact matrix-type pseudo-Riemannian manifolds
IEEE Transactions on Neural Networks
Implementing online natural gradient learning: problems and solutions
IEEE Transactions on Neural Networks
| Hi-index | 0.00 |
Adaptive natural gradient learning avoids singularities in the parameter space of multilayer perceptrons. However, it requires a larger number of additional parameters than ordinary backpropagation in the form of the Fisher information matrix. This article describes a new approach to natural gradient learning that uses a smaller Fisher information matrix. It also uses a prior distribution on the neural network parameters and an annealed learning rate. While this new approach is computationally simpler, its performance is comparable to that of Adaptive natural gradient learning.