Convergence analysis of a class of Hyvärinen-Oja's ICA learning algorithms with constant learning rates

  • Authors:

  • Jian Cheng Lv;Kok Kiong Tan;Zhang Yi;Sunan Huang

  • Affiliations:

  • Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, China and Department of Electrical and Computer Engineering, National University of Singapore, Singapore;Department of Electrical and Computer, National University of Singapore, Singapore;Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, China;Department of Electrical and Computer, National University of Singapore, Singapore

  • Venue:
  • IEEE Transactions on Signal Processing
  • Year:
  • 2009

Quantified Score

Hi-index 35.68

Visualization

Abstract

The convergence of a class of Hyvärinen-Oja's independent component analysis (ICA) learning algorithms with constant learning rates is investigated by analyzing the original stochastic discrete time (SDT) algorithms and the corresponding deterministic discrete time (DDT) algorithms. Most existing learning rates for ICA learning algorithms are required to approach zero as the learning step increases. However, this is not a reasonable requirement to impose in many practical applications. Constant learning rates overcome the shortcoming. Onthe other hand, the original algorithms, described by the SDT algorithms, are studied directly. Invariant sets of these algorithms are obtained so that the nondivergence of the algorithms is guaranteed in stochastic environment. In the invariant sets, the local convergence ofthe original algorithms is analyzed by indirectly studying the convergence of the corresponding DDT algorithms. It is rigorously proven that the trajectories of the DDT algorithms starting from the invariant sets will converge to an independent component direction with a positive kurtosis or a negative kurtosis. The convergence results can shed some light on the dynamical behaviors of the original SDT algorithms. Furthermore, the corresponding DDT algorithms are extended to the block versions of the original SDT algorithms. The block algorithms not only establish a relationship between the SDT algorithms and the corresponding DDT algorithms, but also can get a good convergence speed and accuracy in practice. Simulation examples are carried out to illustrate the theory derived.