Pattern recognition using neural networks: theory and algorithms for engineers and scientists
Pattern recognition using neural networks: theory and algorithms for engineers and scientists
Parameter convergence and learning curves for neural networks
Neural Computation
Iterative solution of nonlinear equations in several variables
Iterative solution of nonlinear equations in several variables
Neural Networks: A Comprehensive Foundation
Neural Networks: A Comprehensive Foundation
Gradient Convergence in Gradient methods with Errors
SIAM Journal on Optimization
Deterministic convergence of an online gradient method for neural networks
Journal of Computational and Applied Mathematics - Selected papers of the international symposium on applied mathematics, August 2000, Dalian, China
Convergence of an online gradient method for feedforward neural networks with stochastic inputs
Journal of Computational and Applied Mathematics - Special issue on proceedings of the international symposium on computational mathematics and applications
Online Learning from Finite Training Sets and Robustness to Input Bias
Neural Computation
Convergence of an online gradient method for BP neural networks with stochastic inputs
ICNC'05 Proceedings of the First international conference on Advances in Natural Computation - Volume Part I
Improving the error backpropagation algorithm with a modified error function
IEEE Transactions on Neural Networks
Deterministic convergence of an online gradient method for BP neural networks
IEEE Transactions on Neural Networks
Convergence of gradient method with momentum for two-Layer feedforward neural networks
IEEE Transactions on Neural Networks
| Hi-index | 0.00 |
The backpropogation (BP) neural networks have been widely applied in scientific research and engineering. The success of the application, however, relies upon the convergence of the training procedure involved in the neural network learning. We settle down the convergence analysis issue through proving two fundamental theorems on the convergence of the online BP training procedure. One theorem claims that under mild conditions, the gradient sequence of the error function will converge to zero (the weak convergence), and another theorem concludes the convergence of the weight sequence defined by the procedure to a fixed value at which the error function attains its minimum (the strong convergence). The weak convergence theorem sharpens and generalizes the existing convergence analysis conducted before, while the strong convergence theorem provides new analysis results on convergence of the online BP training procedure. The results obtained reveal that with any analytic sigmoid activation function, the online BP training procedure is always convergent, which then underlies successful application of the BP neural networks.