Compressed matrix multiplication

• Authors: Rasmus Pagh
• Venue: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference
• Year: 2012

Motivated by the problems of computing sample covariance matrices, and of transforming a collection of vectors to a basis where they are sparse, we present a simple algorithm that computes an approximation of the product of two n-by-n real matrices A and B. Let ||AB||F denote the Frobenius norm of AB, and b be a parameter determining the time/accuracy trade-off. Given 2-wise independent hash functions h1, h2: [n] → [b], and s1, s2: [n] → {-1, +1} the algorithm works by first "compressing" the matrix product into the polynomial [EQUATION] Using FFT for polynomial multiplication, we can compute c0,...,cb-1 such that Σi cixi = (p(x) mod xb) + (p(x) div xb) in time Õ(n2 + nb). An unbiased estimator of (AB)ij with variance at most ||AB||2F/b can then be computed as: [EQUATION] Our approach also leads to an algorithm for computing AB exactly, whp., in time Õ(N + nb) in the case where A and B have at most N nonzero entries, and AB has at most b nonzero entries. Also, we use error-correcting codes in a novel way to recover significant entries of AB in near-linear time.