On Model Selection Consistency of Lasso
The Journal of Machine Learning Research
On the Consistency of Feature Selection using Greedy Least Squares Regression
The Journal of Machine Learning Research
IEEE Transactions on Information Theory
On recovery of sparse signals via l1 minimization
IEEE Transactions on Information Theory
Analysis of Multi-stage Convex Relaxation for Sparse Regularization
The Journal of Machine Learning Research
Greed is good: algorithmic results for sparse approximation
IEEE Transactions on Information Theory
Decoding by linear programming
IEEE Transactions on Information Theory
Stable recovery of sparse overcomplete representations in the presence of noise
IEEE Transactions on Information Theory
Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise
IEEE Transactions on Information Theory
Adaptive Forward-Backward Greedy Algorithm for Learning Sparse Representations
IEEE Transactions on Information Theory
Nonconcave Penalized Likelihood With NP-Dimensionality
IEEE Transactions on Information Theory
Sparse Recovery With Orthogonal Matching Pursuit Under RIP
IEEE Transactions on Information Theory
| Hi-index | 0.00 |
We consider the following sparse signal recovery (or feature selection) problem: given a design matrix X ∈ Rn×m (m ≫ n) and a noisy observation vector y ∈ Rn satisfying y = Xβ* + ε where ε is the noise vector following a Gaussian distribution N(0,σ2I), how to recover the signal (or parameter vector) β* when the signal is sparse? The Dantzig selector has been proposed for sparse signal recovery with strong theoretical guarantees. In this paper, we propose a multi-stage Dantzig selector method, which iteratively refines the target signal β*. We show that if X obeys a certain condition, then with a large probability the difference between the solution β estimated by the proposed method and the true solution β* measured in terms of the lp norm (p ≥ 1) is bounded as ||β-β*||p ≤ (C(s-N)1/p√log m + Δ)σ, where C is a constant, s is the number of nonzero entries in β*, the risk of the oracle estimator Δ is independent of m and is much smaller than the first term, and N is the number of entries of β* larger than a certain value in the order of O(σ√log m). The proposed method improves the estimation bound of the standard Dantzig selector approximately from Cs1/p √logmσ to C(s-N)1/p√logmσ where the value N depends on the number of large entries in β*. When N = s, the proposed algorithm achieves the oracle solution with a high probability, where the oracle solution is the projection of the observation vector y onto true features. In addition, with a large probability, the proposed method can select the same number of correct features under a milder condition than the Dantzig selector. Finally, we extend this multi-stage procedure to the LASSO case.