A randomized solver for linear systems with exponential convergence

Authors:
Thomas Strohmer;Roman Vershynin
Affiliations:
Department of Mathematics, University of California, Davis, CA;Department of Mathematics, University of California, Davis, CA
Venue:
APPROX'06/RANDOM'06 Proceedings of the 9th international conference on Approximation Algorithms for Combinatorial Optimization Problems, and 10th international conference on Randomization and Computation
Year:
2006

Citing 3
Cited 2

The rate of convergence of conjugate gradients

Numerische Mathematik
Eigenvalues and condition numbers of random matrices

SIAM Journal on Matrix Analysis and Applications
Fast monte-carlo algorithms for finding low-rank approximations

Journal of the ACM (JACM)

GPU computing with Kaczmarz's and other iterative algorithms for linear systems

Parallel Computing
Acceleration of randomized Kaczmarz method via the Johnson---Lindenstrauss Lemma

Numerical Algorithms

Quantified Score

Hi-index	0.00

Visualization

Abstract

The Kaczmarz method for solving linear systems of equations Ax=b is an iterative algorithm that has found many applications ranging from computer tomography to digital signal processing. Despite the popularity of this method, useful theoretical estimates for its rate of convergence are still scarce. We introduce a randomized version of the Kaczmarz method for overdetermined linear systems and we prove that it converges with expected exponential rate. Furthermore, this is the first solver whose rate does not depend on the number of equations in the system. The solver does not even need to know the whole system, but only its small random part. It thus outperforms all previously known methods on extremely overdetermined systems. Even for moderately overdetermined systems, numerical simulations reveal that our algorithm can converge faster than the celebrated conjugate gradient algorithm.