Convergence of Gradient Descent Algorithm for a Recurrent Neuron

Authors:
Dongpo Xu;Zhengxue Li;Wei Wu;Xiaoshuai Ding;Di Qu
Affiliations:
Dept. Appl. Math., Dalian University of Technology, Dalian 116023, P.R. China;Dept. Appl. Math., Dalian University of Technology, Dalian 116023, P.R. China;Dept. Appl. Math., Dalian University of Technology, Dalian 116023, P.R. China;Dept. Appl. Math., Dalian University of Technology, Dalian 116023, P.R. China;Dept. Appl. Math., Dalian University of Technology, Dalian 116023, P.R. China
Venue:
ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks, Part III
Year:
2007

Citing 6
Cited 0

Iterative solution of nonlinear equations in several variables

Iterative solution of nonlinear equations in several variables
Neutral Networks in Optimization

Neutral Networks in Optimization
Neural Networks: A Comprehensive Foundation

Neural Networks: A Comprehensive Foundation
A learning algorithm for continually running fully recurrent neural networks

Neural Computation
A convergence result for learning in recurrent neural networks

Neural Computation
Optimal convergence of on-line backpropagation

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

Probabilistic convergence results of online gradient descent algorithm have been obtained by many authors for the training of recurrent neural networks with innitely many training samples. This paper proves deterministic convergence of o2ine gradient descent algorithm for a recurrent neural network with nite number of training samples. Our results can be hopefully extended to more complicated recurrent neural networks, and serve as a complementary result to the existing probability convergence results.