Global Convergence of GHA Learning Algorithm With Nonzero-Approaching Adaptive Learning Rates

Authors:
Jian Cheng Lv;Zhang Yi;Kok Kiong Tan
Affiliations:
Univ. of Electron. Sci. & Technol. of China, Chengdu;-;-
Venue:
IEEE Transactions on Neural Networks
Year:
2007

Citing 0
Cited 3

Stability and Chaos of a Class of Learning Algorithms for ICA Neural Networks

Neural Processing Letters
Convergence analysis of a class of Hyvärinen-Oja's ICA learning algorithms with constant learning rates

IEEE Transactions on Signal Processing
A family of fuzzy learning algorithms for robust principal component analysis neural networks

IEEE Transactions on Fuzzy Systems

Quantified Score

Hi-index	0.01

Visualization

Abstract

The generalized Hebbian algorithm (GHA) is one of the most widely used principal component analysis (PCA) neural network (NN) learning algorithms. Learning rates of GHA play important roles in convergence of the algorithm for applications. Traditionally, the learning rates of GHA are required to converge to zero so that its convergence can be analyzed by studying the corresponding deterministic continuous-time (DCT) equations. However, the requirement for learning rates to approach zero is not a practical one in applications due to computational roundoff limitations and tracking requirements. In this paper, nonzero-approaching adaptive learning rates are proposed to overcome this problem. These proposed adaptive learning rates converge to some positive constants, which not only speed up the algorithm evolution considerably, but also guarantee global convergence of the GHA algorithm. The convergence is studied in detail by analyzing the corresponding deterministic discrete-time (DDT) equations. Extensive simulations are carried out to illustrate the theory.