Soft-competitive learning of sparse codes and its application to image reconstruction

Authors:
Kai Labusch;Erhardt Barth;Thomas Martinetz
Affiliations:
University of Lübeck, Institute for Neuro- and Bioinformatics, Ratzeburger Allee 160, 23538 Lübeck, Germany;University of Lübeck, Institute for Neuro- and Bioinformatics, Ratzeburger Allee 160, 23538 Lübeck, Germany;University of Lübeck, Institute for Neuro- and Bioinformatics, Ratzeburger Allee 160, 23538 Lübeck, Germany
Venue:
Neurocomputing
Year:
2011

Citing 22
Cited 1

Learning nonlinear overcomplete representations for efficient coding

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Atomic Decomposition by Basis Pursuit

SIAM Review
Dictionary learning algorithms for sparse representation

Neural Computation
Learning Overcomplete Representations

Neural Computation
A simple test to check the optimality of a sparse signal approximation

Signal Processing - Sparse approximations in signal and image processing
Batch and median neural gas

Neural Networks - 2006 Special issue: Advances in self-organizing maps--WSOM'05
Method of optimal directions for frame design

ICASSP '99 Proceedings of the Acoustics, Speech, and Signal Processing, 1999. on 1999 IEEE International Conference - Volume 05
Sparse coding via thresholding and local competition in neural circuits

Neural Computation
Sparse Coding Neural Gas: Learning of overcomplete data representations

Neurocomputing
A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way

A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images

SIAM Review
-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation

IEEE Transactions on Signal Processing
Subset selection in noise based on diversity measure minimization

IEEE Transactions on Signal Processing
Matching pursuits with time-frequency dictionaries

IEEE Transactions on Signal Processing
Greed is good: algorithmic results for sparse approximation

IEEE Transactions on Information Theory
Stable recovery of sparse overcomplete representations in the presence of noise

IEEE Transactions on Information Theory
Compressed sensing

IEEE Transactions on Information Theory
Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

IEEE Transactions on Information Theory
Adaptive wavelet thresholding for image denoising and compression

IEEE Transactions on Image Processing
Image coding using wavelet transform

IEEE Transactions on Image Processing
Sparse Representation for Color Image Restoration

IEEE Transactions on Image Processing
`Neural-gas' network for vector quantization and its application to time-series prediction

IEEE Transactions on Neural Networks

Applying alternating direction method of multipliers for constrained dictionary learning

Neurocomputing

Quantified Score

Hi-index	0.01

Visualization

Abstract

We propose a new algorithm for the design of overcomplete dictionaries for sparse coding, neural gas for dictionary learning (NGDL), which uses a set of solutions for the sparse coefficients in each update step of the dictionary. In order to obtain such a set of solutions, we additionally propose the bag of pursuits (BOP) method for sparse approximation. Using BOP in order to determine the coefficients of the dictionary, we show in an image encoding experiment that in case of limited training data and limited computation time the NGDL update of the dictionary performs better than the standard gradient approach that is used for instance in the Sparsenet algorithm, or other state-of-the-art methods for dictionary learning such as the method of optimal directions (MOD) or the widely used K-SVD algorithm. In an application to image reconstruction, dictionaries trained with this algorithm outperform not only overcomplete Haar-wavelets and overcomplete discrete cosine transformations, but also dictionaries obtained with widely used algorithms like K-SVD.