Closed-form MMSE estimation for signal denoising under sparse representation modeling over a unitary dictionary

Authors:
Matan Protter;Irad Yavneh;Michael Elad
Affiliations:
Computer Science Department, Technion-Israel Institute of Technology, Haifa, Israel;Computer Science Department, Technion-Israel Institute of Technology, Haifa, Israel;Computer Science Department, Technion-Israel Institute of Technology, Haifa, Israel
Venue:
IEEE Transactions on Signal Processing
Year:
2010

Citing 11
Cited 3

Sparse Approximate Solutions to Linear Systems

SIAM Journal on Computing
Atomic Decomposition by Basis Pursuit

SIAM Review
Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity

Machine Learning
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images

SIAM Review
A plurality of sparse representations is better than the sparsest one alone

IEEE Transactions on Information Theory
Linear Regression With a Sparse Parameter Vector

IEEE Transactions on Signal Processing
-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation

IEEE Transactions on Signal Processing
Analysis of multiresolution image denoising schemes using generalized Gaussian and complexity priors

IEEE Transactions on Information Theory
Why Simple Shrinkage Is Still Relevant for Redundant Representations?

IEEE Transactions on Information Theory
Wavelet-based texture retrieval using generalized Gaussian density and Kullback-Leibler distance

IEEE Transactions on Image Processing
Image quality assessment: from error visibility to structural similarity

IEEE Transactions on Image Processing

An MMSE approach to nonlocal image denoising: Theory and practical implementation

Journal of Visual Communication and Image Representation
Searching for the best matching atoms based on multi-swarm co-operative PSO

IScIDE'11 Proceedings of the Second Sino-foreign-interchange conference on Intelligent Science and Intelligent Data Engineering
On MAP and MMSE estimators for the co-sparse analysis model

Digital Signal Processing

Quantified Score

Hi-index	35.68

Visualization

Abstract

This paper deals with the Bayesian signal denoising problem, assuming a prior based on a sparse representation modeling over a unitary dictionary. It is well known that the maximum a posteriori probability (MAP) estimator in such a case has a closed-form solution based on a simple shrinkage. The focus in this paper is on the better performing and less familiar minimummean-squared-error (MMSE) estimator. We show that this estimator also leads to a simple formula, in the form of a plain recursive expression for evaluating the contribution of every atom in the solution. An extension of the model to real-world signals is also offered, considering heteroscedastic nonzero entries in the representation, and allowing varying probabilities for the chosen atoms and the overall cardinality of the sparse representation. The MAP and MMSE estimators are redeveloped for this extended model, again resulting in closed-form simple algorithms. Finally, the superiority of the MMSE estimator is demonstrated both on synthetically generated signals and on real-world signals (image patches).