An adaptive penalized maximum likelihood algorithm

Authors:
Guang Deng;Wai-Yin Ng
Affiliations:
Department of Electronic Engineering, La Trobe University, Bundoora, Victoria 3086, Australia;Department of Information Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong
Venue:
Signal Processing
Year:
2008

Citing 11
Cited 0

Adaptive filter theory (2nd ed.)

Adaptive filter theory (2nd ed.)
Bayesian interpolation

Neural Computation
The least mean square family

Adaptive system identification and signal processing algorithms
Natural gradient works efficiently in learning

Neural Computation
Adaptive Sparseness for Supervised Learning

IEEE Transactions on Pattern Analysis and Machine Intelligence
Convex Optimization

Convex Optimization
Exploiting sparsity in adaptive filters

IEEE Transactions on Signal Processing
Complexity reduction of the NLMS algorithm via selectivecoefficient update

IEEE Transactions on Signal Processing
A recursive least M-estimate algorithm for robust adaptive filtering in impulsive noise: fast algorithm and convergence performance analysis

IEEE Transactions on Signal Processing
A variable step size LMS algorithm

IEEE Transactions on Signal Processing
A robust variable step-size LMS-type algorithm: analysis andsimulations

IEEE Transactions on Signal Processing

Quantified Score

Hi-index	0.08

Visualization

Abstract

The LMS algorithm is one of the most popular learning algorithms for identifying an unknown system. Many variants of the algorithm have been developed based on different problem formulations and principles. In this paper, we use the penalized maximum likelihood (PML) as a principled and unified approach for developing LMS-type algorithms. We study a general solution to the problem and develop algorithms to address the problems of robustness to impulsive noise and exploiting the sparseness of the system. We perform a statistical analysis of a special case of the proposed algorithm and propose a data-driven method to update the penalty parameter. We also reveal an invariant property of the algorithm. Connections with algorithms based on stochastic gradient descent are also studied. We demonstrate the competitive performance of the proposed algorithms by numerical examples and comparison with recently published algorithms.