Asymptotically optimal smoothing of averaged LMS estimates for regression parameter tracking

Authors:
Alexander V. Nazin;Lennart Ljung
Affiliations:
Institute of Control Sciences, Profsoyuznaya str., 65, 117997 Moscow, Russia;Department of Electrical Engineering, Linköping University, SE-581 83 Linköping, Sweden
Venue:
Automatica (Journal of IFAC)
Year:
2002

Citing 3
Cited 0

Adaptation and tracking in system identification—a survey

Automatica (Journal of IFAC) - Identification and system parameter estimation
Acceleration of stochastic approximation by averaging

SIAM Journal on Control and Optimization
Stochastic approximation with averaging of the iterates: optimal asymptotic rate of convergence for general processes

SIAM Journal on Control and Optimization

Quantified Score

Hi-index	22.14

Visualization

Abstract

The sequence of estimates formed by the LMS algorithm for a standard linear regression estimation problem is considered. It is known since earlier that smoothing these estimates by simple averaging will lead to, asymptotically, the recursive least-squares algorithm. In this paper, it is first shown that smoothing the LMS estimates using a matrix updating will lead to smoothed estimates with optimal tracking properties, also in case the true parameters are slowly changing as a random walk. The choice of smoothing matrix should be tailored to the properties of the random walk. Second, it is shown that the same accuracy can be obtained also for a modified algorithm, SLAMS, which is based on averages and requires much less computations.