Optimal column-based low-rank matrix reconstruction
Proceedings of the twenty-third annual ACM-SIAM symposium on Discrete Algorithms
Randomized Algorithms for Matrices and Data
Foundations and Trends® in Machine Learning
| Hi-index | 0.00 |
Given a matrix A ε ℝ m×n of rank r, and an integerk r, the top k singular vectorsprovide the best rank-k approximation to A. Whenthe columns of A have specific meaning, it is desirable tofind (provably) "good" approximations to Ak which use only a small number of columns inA. Proposed solutions to this problem have thus farfocused on randomized algorithms. Our main result is a simplegreedy deterministic algorithm with guarantees on the performanceand the number of columns chosen. Specifically, our greedyalgorithm chooses c columns from A with $c=O\left({{k^2\log k} \over {\epsilon^2}}\mu^2(A)\ln\left({\sqrt{k}\|{A_k}\|_F} \over{\epsilon}\|{A-A_k}\|_F\right)\right)$ such that$${\|A-C_{gr}C_{gr}^+A\|}_F \leq \left(1+\epsilon\right)\|{A-A_k}_F,$$where C gr is the matrix composedof the c columns, $C_{gr}^+$ is the pseudo-inverse ofC gr ($C_{gr}C_{gr}^+A$ is the bestreconstruction of A from Cgr), and ¼(A) is ameasure of the coherence in the normalized columns ofA. The running time of the algorithm isO(SVD(A k) +mnc) where SVD(A k)is the running time complexity of computing the first ksingular vectors of A. To the best of our knowledge, thisis the first deterministic algorithm with performance guarantees onthe number of columns and a (1 + ε) approximationratio in Frobenius norm. The algorithm is quite simple andintuitive and is obtained by combining a generalization of the wellknown sparse approximation problem from information theorywith an existence result on the possibility of sparseapproximation. Tightening the analysis along either of these twodimensions would yield improved results.