The nature of statistical learning theory
The nature of statistical learning theory
Natural gradient works efficiently in learning
Neural Computation
On-line learning and stochastic approximations
On-line learning in neural networks
Neural Networks for Pattern Recognition
Neural Networks for Pattern Recognition
Statistical Analysis of Regularization Constant - From Bayes, MDL and NIC Points of View
IWANN '97 Proceedings of the International Work-Conference on Artificial and Natural Neural Networks: Biological and Artificial Computation: From Neuroscience to Technology
Practical Consideration on Generalization Property of Natural Gradient Learning
IWANN '01 Proceedings of the 6th International Work-Conference on Artificial and Natural Neural Networks: Connectionist Models of Neurons, Learning Processes and Artificial Intelligence-Part I
Adaptive improved natural gradient algorithm for blind source separation
Neural Computation
A simplified natural gradient learning algorithm
Advances in Artificial Neural Systems
| Hi-index | 0.00 |
Natural gradient learning is known to be efficient in escaping plateau, which is a main cause of the slow learning speed of neural networks. The adaptive natural gradient learning method for practical implementation also has been developed, and its advantage in real-world problems has been confirmed. In this letter, we deal with the generalization performances of the natural gradient method. Since natural gradient learning makes parameters fit to training data quickly,the overfitting phenomenon may easily occur, which results in poor generalization performance. To solve the problem, we introduce the regularization term in natural gradient learning and propose an efficient optimizing method for the scale of regularization by using a generalized Akaike information criterion (network information criterion). We discuss the properties of the optimized regularization strength by NIC through theoretical analysis as well as computer simulations. We confirm the computational efficiency and generalization performance of the proposed method in real-world applications through computational experiments on benchmark problems.