Deterministic convergence of an online gradient method with momentum

Authors:
Naimin Zhang
Affiliations:
School of Mathematics and Information Science, Wenzhou University, Wenzhou, P.R. China
Venue:
ICIC'06 Proceedings of the 2006 international conference on Intelligent Computing - Volume Part I
Year:
2006

Citing 9
Cited 3

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations

Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations
On the convergence of the LMS algorithm with adaptive learning rate for linear feedforward networks

Neural Computation
Online learning from finite training sets and robustness to input bias

Neural Computation
Fundamentals of Artificial Neural Networks

Fundamentals of Artificial Neural Networks
Deterministic convergence of an online gradient method for neural networks

Journal of Computational and Applied Mathematics - Selected papers of the international symposium on applied mathematics, August 2000, Dalian, China
Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method

Neural Networks
Convergence of an online gradient method for feedforward neural networks with stochastic inputs

Journal of Computational and Applied Mathematics - Special issue on proceedings of the international symposium on computational mathematics and applications
Stability of steepest descent with momentum for quadratic functions

IEEE Transactions on Neural Networks
Convergence of gradient method with momentum for two-Layer feedforward neural networks

IEEE Transactions on Neural Networks

Boundedness and convergence of online gradient method with penalty and momentum

Neurocomputing
Convergence of an online gradient method with inner-product penalty and adaptive momentum

Neurocomputing
Computational properties of cyclic and almost-cyclic learning with momentum for feedforward neural networks

ISNN'12 Proceedings of the 9th international conference on Advances in Neural Networks - Volume Part I

Quantified Score

Hi-index	0.00

Visualization

Abstract

An online gradient method with momentum for feedforward neural network is considered. The learning rate is set to be a constant and the momentum coefficient an adaptive variable. Both the weak and strong convergence results are proved, as well as the convergence rates for the error function and for the weight.