Further results on stable recovery of sparse overcomplete representations in the presence of noise
IEEE Transactions on Information Theory
Analysis and Generalizations of the Linearized Bregman Method
SIAM Journal on Imaging Sciences
SIAM Journal on Imaging Sciences
Matrix-free interior point method
Computational Optimization and Applications
SIAM Journal on Optimization
Error Forgetting of Bregman Iteration
Journal of Scientific Computing
Accelerated Linearized Bregman Method
Journal of Scientific Computing
Hi-index | 0.06 |
The regularization of a convex program is exact if all solutions of the regularized problem are also solutions of the original problem for all values of the regularization parameter below some positive threshold. For a general convex program, we show that the regularization is exact if and only if a certain selection problem has a Lagrange multiplier. Moreover, the regularization parameter threshold is inversely related to the Lagrange multiplier. We use this result to generalize an exact regularization result of Ferris and Mangasarian [Appl. Math. Optim., 23 (1991), pp. 266-273] involving a linearized selection problem. We also use it to derive necessary and sufficient conditions for exact penalization, similar to those obtained by Bertsekas [Math. Programming, 9 (1975), pp. 87-99] and by Bertsekas, Nedić, and Ozdaglar [Convex Analysis and Optimization, Athena Scientific, Belmont, MA, 2003]. When the regularization is not exact, we derive error bounds on the distance from the regularized solution to the original solution set. We also show that existence of a “weak sharp minimum” is in some sense close to being necessary for exact regularization. We illustrate the main result with numerical experiments on the $\ell_1$ regularization of benchmark (degenerate) linear programs and semidefinite/second-order cone programs. The experiments demonstrate the usefulness of $\ell_1$ regularization in finding sparse solutions.