An improved approximation algorithm for the column subset selection problem
SODA '09 Proceedings of the twentieth Annual ACM-SIAM Symposium on Discrete Algorithms
Numerical linear algebra in the streaming model
Proceedings of the forty-first annual ACM symposium on Theory of computing
PCM'10 Proceedings of the Advances in multimedia information processing, and 11th Pacific Rim conference on Multimedia: Part II
A Randomized Algorithm for Principal Component Analysis
SIAM Journal on Matrix Analysis and Applications
Blendenpik: Supercharging LAPACK's Least-Squares Solver
SIAM Journal on Scientific Computing
Acceleration of randomized Kaczmarz method via the Johnson---Lindenstrauss Lemma
Numerical Algorithms
Column subset selection via sparse approximation of SVD
Theoretical Computer Science
Randomized Algorithms for Matrices and Data
Foundations and Trends® in Machine Learning
Sampling methods for the Nyström method
The Journal of Machine Learning Research
Sparsely precomputing the light transport matrix for real-time rendering
EGSR'10 Proceedings of the 21st Eurographics conference on Rendering
Surveillance video coding via low-rank and sparse decomposition
Proceedings of the 20th ACM international conference on Multimedia
Simple and deterministic matrix sketching
Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Fast approximation of matrix coherence and statistical leverage
The Journal of Machine Learning Research
Inverse bi-scale material design
ACM Transactions on Graphics (TOG)
A scalable approach to column-based low-rank matrix approximation
IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Improving CUR matrix decomposition and the Nyström approximation via adaptive sampling
The Journal of Machine Learning Research
Column Subset Selection Problem is UG-hard
Journal of Computer and System Sciences
Hi-index | 0.00 |
Many data analysis applications deal with large matrices and involve approximating the matrix using a small number of “components.” Typically, these components are linear combinations of the rows and columns of the matrix, and are thus difficult to interpret in terms of the original features of the input data. In this paper, we propose and study matrix approximations that are explicitly expressed in terms of a small number of columns and/or rows of the data matrix, and thereby more amenable to interpretation in terms of the original data. Our main algorithmic results are two randomized algorithms which take as input an $m\times n$ matrix $A$ and a rank parameter $k$. In our first algorithm, $C$ is chosen, and we let $A'=CC^+A$, where $C^+$ is the Moore-Penrose generalized inverse of $C$. In our second algorithm $C$, $U$, $R$ are chosen, and we let $A'=CUR$. ($C$ and $R$ are matrices that consist of actual columns and rows, respectively, of $A$, and $U$ is a generalized inverse of their intersection.) For each algorithm, we show that with probability at least $1-\delta$, $\|A-A'\|_F\leq(1+\epsilon)\,\|A-A_k\|_F$, where $A_k$ is the “best” rank-$k$ approximation provided by truncating the SVD of $A$, and where $\|X\|_F$ is the Frobenius norm of the matrix $X$. The number of columns of $C$ and rows of $R$ is a low-degree polynomial in $k$, $1/\epsilon$, and $\log(1/\delta)$. Both the Numerical Linear Algebra community and the Theoretical Computer Science community have studied variants of these matrix decompositions over the last ten years. However, our two algorithms are the first polynomial time algorithms for such low-rank matrix approximations that come with relative-error guarantees; previously, in some cases, it was not even known whether such matrix decompositions exist. Both of our algorithms are simple and they take time of the order needed to approximately compute the top $k$ singular vectors of $A$. The technical crux of our analysis is a novel, intuitive sampling method we introduce in this paper called “subspace sampling.” In subspace sampling, the sampling probabilities depend on the Euclidean norms of the rows of the top singular vectors. This allows us to obtain provable relative-error guarantees by deconvoluting “subspace” information and “size-of-$A$” information in the input matrix. This technique is likely to be useful for other matrix approximation and data analysis problems.