A modified Armijo rule for the online selection of learning rate of the LMS algorithm

  • Authors:
  • C. Boukis;D. P. Mandic;A. G. Constantinides;L. C. Polymenakos

  • Affiliations:
  • Athens Information Technology, Autonomic and Grid Computing Group, Markopoulo Ave, Peania/Athens 19002, Greece;Imperial College, Electrical and Electronic Engineering Department, Communications and Signal Processing Group, Exhibition Rd, London SW7 2BT, UK;Imperial College, Electrical and Electronic Engineering Department, Communications and Signal Processing Group, Exhibition Rd, London SW7 2BT, UK;Athens Information Technology, Autonomic and Grid Computing Group, Markopoulo Ave, Peania/Athens 19002, Greece

  • Venue:
  • Digital Signal Processing
  • Year:
  • 2010

Quantified Score

Hi-index 0.01

Visualization

Abstract

The use of the Armijo rule for the automatic selection of the step size within the class of stochastic gradient descent algorithms is investigated, and the Armijo rule learning rate least mean square (ALR-LMS) algorithm is introduced. This algorithm is derived by integrating an appropriately modified version of the Armijo line search to the least mean square filter update. The analysis of the stability, robustness and the bounds on the parameters which guarantee convergence is conducted, and some practical issues relating the choice of parameters of the ALR-LMS and computational complexity are addressed. Comprehensive simulation results in the system identification and prediction setting support the analysis.