Lossy source compression using low-density generator matrix codes: analysis and algorithms

  • Authors:
  • Martin J. Wainwright;Elitza Maneva;Emin Martinian

  • Affiliations:
  • Department of Statistics, and Department of Electrical Engineering and Computer Sciences, University of California at Berkeley, Berkeley, CA;Institut de Matematica, Universitat de Barcelona, Barcelona, Spain;Tilda Consulting, Boston, MA

  • Venue:
  • IEEE Transactions on Information Theory
  • Year:
  • 2010

Quantified Score

Hi-index 754.84

Visualization

Abstract

We study the use of low-density generator matrix (LDGM) codes for lossy compression of the Bernoulli symmetric source. First, we establish rigorous upper bounds on the average distortion achieved by check-regular ensemble of LDGM codes under optimal minimum distance source encoding. These bounds establish that the average distortion using such bounded degree families rapidly approaches the Shannon limit as the degrees are increased. Second, we propose a family of message-passing algorithms, ranging from the standard belief propagation algorithm at one extreme to a variant of survey propagation algorithm at the other. When combined with a decimation subroutine and applied to LDGM codes with suitably irregular degree distributions, we show that such a message-passing/decimation algorithm yields distortion very close to the Shannon rate-distortion bound for the binary symmetric source.