On the stability of the basis pursuit in the presence of noise

  • Authors:
  • David L. Donoho;Michael Elad

  • Affiliations:
  • Department of Statistics, Stanford University, Stanford, CA;Department of Computer Science, The Technion-Israel Institute of Technology, Haifa, Israel

  • Venue:
  • Signal Processing - Sparse approximations in signal and image processing
  • Year:
  • 2006

Quantified Score

Hi-index 0.06

Visualization

Abstract

Given a signal S ∈ RN and a full-rank matrix D ∈ RN × L with N , we define the signal's overcomplete representation as α ∈ RL satisfying S= Dα. Among the infinitely many solutions of this under-determined linear system of equations, we have special interest in the sparsest representation, i.e., the one minimizing ||α||0. This problem has a combinatorial flavor to it, and its direct solution is impossible even for moderate L. Approximation algorithms are thus required, and one such appealing technique is the basis pursuit (BP) algorithm. This algorithm has been the focus of recent theoretical research effort. It was found that if indeed the representation is sparse enough, BP finds it accurately.When an error is permitted in the composition of the signal, we no longer require exact equality S=Dα. The BP has been extended to treat this case, leading to a denoizing algorithm. The natural question to pose is how the above-mentioned theoretical results generalize to this more practical mode of operation. In this paper we propose such a generalization. The behavior of the basis pursuit in the presence of noise has been the subject of two independent very wide contributions released for publication very recently. This paper is another contribution in this direction, but as opposed to the others mentioned, this paper aims to present a somewhat simplified picture of the topic, and thus could be referred to as a primer to this field. Specifically, we establish here the stability of the BP in the presence of noise for sparse enough representations. We study both the case of a general dictionary D, and a special case where D is built as a union of orthonormal bases. This work is a direct generalization of noiseless BP study, and indeed, when the noise power is reduced to zero, we obtain the known results of the noiseless BP.