Learning Sparse Overcomplete Codes for Images

  • Authors:
  • Joseph F. Murray;Kenneth Kreutz-Delgado

  • Affiliations:
  • Brain and Cognitive Sciences Department, Massachusetts Institute of Technology, Cambridge, USA 02139;Electrical and Computer Engineering Department, University of California, San Diego, La Jolla, USA 92093-0407

  • Venue:
  • Journal of VLSI Signal Processing Systems
  • Year:
  • 2006

Quantified Score

Hi-index 0.00

Visualization

Abstract

Images can be coded accurately using a sparse set of vectors from a learned overcomplete dictionary, with potential applications in image compression and feature selection for pattern recognition. We present a survey of algorithms that perform dictionary learning and sparse coding and make three contributions. First, we compare our overcomplete dictionary learning algorithm (FOCUSS-CNDL) with overcomplete independent component analysis (ICA). Second, noting that once a dictionary has been learned in a given domain the problem becomes one of choosing the vectors to form an accurate, sparse representation, we compare a recently developed algorithm (sparse Bayesian learning with adjustable variance Gaussians, SBL-AVG) to well known methods of subset selection: matching pursuit and FOCUSS. Third, noting that in some cases it may be necessary to find a non-negative sparse coding, we present a modified version of the FOCUSS algorithm that can find such non-negative codings. Efficient parallel implementations in VLSI could make these algorithms more practical for many applications.