Fast computation of low-rank matrix approximations

  • Authors:
  • Dimitris Achlioptas;Frank Mcsherry

  • Affiliations:
  • University of California at Santa Cruz, Santa Cruz, California;Microsoft Research, Mountain View, California

  • Venue:
  • Journal of the ACM (JACM)
  • Year:
  • 2007

Quantified Score

Hi-index 0.07

Visualization

Abstract

Given a matrix A, it is often desirable to find a good approximation to A that has low rank. We introduce a simple technique for accelerating the computation of such approximations when A has strong spectral features, that is, when the singular values of interest are significantly greater than those of a random matrix with size and entries similar to A. Our technique amounts to independently sampling and/or quantizing the entries of A, thus speeding up computation by reducing the number of nonzero entries and/or the length of their representation. Our analysis is based on observing that the acts of sampling and quantization can be viewed as adding a random matrix N to A, whose entries are independent random variables with zero-mean and bounded variance. Since, with high probability, N has very weak spectral features, we can prove that the effect of sampling and quantization nearly vanishes when a low-rank approximation to A + N is computed. We give high probability bounds on the quality of our approximation both in the Frobenius and the 2-norm.