A Variational Method for Learning Sparse and Overcomplete Representations

Authors:
Mark Girolami
Affiliations:
Laboratory of Computing and Information Science, Helsinki University of Technology, Finland
Venue:
Neural Computation
Year:
2001

Citing 7
Cited 36

An information-maximization approach to blind separation and blind deconvolution

Neural Computation
Image Representation Using 2D Gabor Wavelets

IEEE Transactions on Pattern Analysis and Machine Intelligence
Independent factor analysis

Neural Computation
ICA Mixture Models for Unsupervised Classification of Non-Gaussian Classes and Automatic Context Switching in Blind Signal Separation

IEEE Transactions on Pattern Analysis and Machine Intelligence
Variational methods for inference and estimation in graphical models

Variational methods for inference and estimation in graphical models
Blind Source Separation by Sparse Decomposition in a Signal Dictionary

Neural Computation
Learning Overcomplete Representations

Neural Computation

Dictionary learning algorithms for sparse representation

Neural Computation
Energy-based models for sparse overcomplete representations

The Journal of Machine Learning Research
Hierarchical models of variance sources

Signal Processing - Special issue on independent components analysis and beyond
Analysis of sparse representation and blind source separation

Neural Computation
Energy-based models for sparse overcomplete representations

The Journal of Machine Learning Research
Nonlinear and Noisy Extension of Independent Component Analysis: Theory and Its Application to a Pitch Sensation Model

Neural Computation
Blind source separation of more sources than mixtures using sparse mixture models

Pattern Recognition Letters
A Gaussian mixture model for underdetermined independent component analysis

Signal Processing
Learning Sparse Overcomplete Codes for Images

Journal of VLSI Signal Processing Systems
Learning Sparse Overcomplete Codes for Images

Journal of VLSI Signal Processing Systems
Source separation using single channel ICA

Signal Processing
A parametric density model for blind source separation

Neural Processing Letters
Monaural speech segregation based on fusion of source-driven with model-driven techniques

Speech Communication
Variational and stochastic inference for Bayesian source separation

Digital Signal Processing
A null space method for over-complete blind source separation

Computational Statistics & Data Analysis
A maximum likelihood estimation of vocal-tract-related filter characteristics for single channel speech separation

EURASIP Journal on Audio, Speech, and Music Processing
Flexible and efficient implementations of Bayesian independent component analysis

Neurocomputing
A note on lewicki-sejnowski gradient for learning overcomplete representations

Neural Computation
Sparse Bayesian nonparametric regression

Proceedings of the 25th international conference on Machine learning
Bayesian Inference and Optimal Design for the Sparse Linear Model

The Journal of Machine Learning Research
An Efficient K-Hyperplane Clustering Algorithm and Its Application to Sparse Component Analysis

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Part II--Advances in Neural Networks
K-hyperline clustering learning for sparse component analysis

Signal Processing
Convex variational Bayesian inference for large scale generalized linear models

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Independent factor topic models

ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
Underdetermined blind source separation based on subspace representation

IEEE Transactions on Signal Processing
A simple overcomplete ICA algorithm by non-orthogonal pair optimizations

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Gamma Markov random fields for audio source modeling

IEEE Transactions on Audio, Speech, and Language Processing
Bayesian compressive sensing using Laplace priors

IEEE Transactions on Image Processing
Super-Linear Convergence of Dual Augmented Lagrangian Algorithm for Sparsity Regularized Estimation

The Journal of Machine Learning Research
Large Scale Bayesian Inference and Experimental Design for Sparse Linear Models

SIAM Journal on Imaging Sciences
Post-nonlinear underdetermined ICA by bayesian statistics

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Analysis of source sparsity and recoverability for SCA based blind source separation

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Super-Gaussian mixture source model for ICA

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Identification of mixing matrix in blind source separation

ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
Partially linear estimation with application to image deblurring using blurred/noisy image pairs

LVA/ICA'12 Proceedings of the 10th international conference on Latent Variable Analysis and Signal Separation
Gaussian Kullback-Leibler approximate inference

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

An expectation-maximization algorithm for learning sparse and overcomplete data representations is presented. The proposed algorithm exploits a variational approximation to a range of heavy-tailed distributions whose limit is the Laplacian. A rigorous lower bound on the sparse prior distribution is derived, which enables the analytic marginalization of a lower bound on the data likelihood. This lower bound enables the development of an expectation-maximization algorithm for learning the overcomplete basis vectors and inferring the most probable basis coefficients.