Online adaptive estimation of sparse signals: where RLS meets the l1-norm

Authors:
Daniele Angelosante;Juan Andrés Bazerque;Georgios B. Giannakis
Affiliations:
Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN;Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN;Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN
Venue:
IEEE Transactions on Signal Processing
Year:
2010

Citing 8
Cited 2

Adaptive signal processing algorithms: stability and performance

Adaptive signal processing algorithms: stability and performance
Atomic Decomposition by Basis Pursuit

SIAM Journal on Scientific Computing
Convergence of a block coordinate descent method for nondifferentiable minimization

Journal of Optimization Theory and Applications
Sparse LMS for system identification

ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
RLS-weighted Lasso for adaptive estimation of sparse signals

ICASSP '09 Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing
Low-Complexity RLS Algorithms Using Dichotomous Coordinate Descent Iterations

IEEE Transactions on Signal Processing - Part II
Partial update LMS algorithms

IEEE Transactions on Signal Processing
Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?

IEEE Transactions on Information Theory

Compressive system identification: Sequential methods and entropy bounds

Digital Signal Processing
Adaptive matching pursuit using coordinate descent and double residual minimization

Signal Processing

Quantified Score

Hi-index	35.68

Visualization

Abstract

Using the l1-norm to regularize the least-squares criterion, the batch least-absolute shrinkage and selection operator (Lasso) has well-documented merits for estimating sparse signals of interest emerging in various applications where observations adhere to parsimonious linear regression models. To cope with high complexity, increasing memory requirements, and lack of tracking capability that batch Lasso estimators face when processing observations sequentially, the present paper develops a novel time-weighted Lasso (TWL) approach. Performance analysis reveals that TWL cannot estimate consistently the desired signal support without compromising rate of convergence. This motivates the development of a time- and norm-weighted Lasso (TNWL) scheme with l1-norm weights obtained from the recursive least-squares (RLS) algorithm. The resultant algorithm consistently estimates the support of sparse signals without reducing the convergence rate. To cope with sparsity-aware recursive real-time processing, novel adaptive algorithms are also developed to enable online coordinate descent solvers of TWL and TNWL that provably converge to the true sparse signal in the time-invariant case. Simulated tests compare competing alternatives and corroborate the performance of the novel algorithms in estimating time-invariant signals, and tracking time-varying signals under sparsity constraints.