The space complexity of approximating the frequency moments
STOC '96 Proceedings of the twenty-eighth annual ACM symposium on Theory of computing
Atomic Decomposition by Basis Pursuit
SIAM Journal on Scientific Computing
Database-friendly random projections
PODS '01 Proceedings of the twentieth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems
Lectures on Discrete Geometry
An elementary proof of a theorem of Johnson and Lindenstrauss
Random Structures & Algorithms
Learning Mixtures of Gaussians
FOCS '99 Proceedings of the 40th Annual Symposium on Foundations of Computer Science
Convex Optimization
Stable distributions, pseudorandom generators, embeddings, and data stream computation
Journal of the ACM (JACM)
Very sparse random projections
Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Distributed sparse random projections for refinable approximation
Proceedings of the 6th international conference on Information processing in sensor networks
On Model Selection Consistency of Lasso
The Journal of Machine Learning Research
Nonlinear Estimators and Tail Bounds for Dimension Reduction in l1 Using Cauchy Random Projections
The Journal of Machine Learning Research
IEEE Transactions on Information Theory
Information-theoretic limits on sparse signal recovery: dense versus sparse measurement matrices
IEEE Transactions on Information Theory
Decoding by linear programming
IEEE Transactions on Information Theory
Just relax: convex programming methods for identifying sparse signals in noise
IEEE Transactions on Information Theory
LP Decoding Corrects a Constant Fraction of Errors
IEEE Transactions on Information Theory
Consistency of sparse PCA in High Dimension, Low Sample Size contexts
Journal of Multivariate Analysis
| Hi-index | 0.00 |
We consider the problem of high-dimensional variable selection: given n noisy observations of a k-sparse vector β* ∈ Rp, estimate the subset of non-zero entries of β*. A significant body of work has studied behavior of l1-relaxations when applied to random measurement matrices that are dense (e.g., Gaussian, Bernoulli). In this paper, we analyze sparsified measurement ensembles, and consider the trade-off between measurement sparsity, as measured by the fraction γ of non-zero entries, and the statistical efficiency, as measured by the minimal number of observations n required for correct variable selection with probability converging to one. Our main result is to prove that it is possible to let the fraction on non-zero entries γ → 0 at some rate, yielding measurement matrices with a vanishing fraction of non-zeros per row, while retaining the same statistical efficiency as dense ensembles. A variety of simulation results confirm the sharpness of our theoretical predictions.