A two-layer model of natural stimuli estimated with score matching

Authors:
Urs Köster;Aapo Hyvärinen
Affiliations:
-;-
Venue:
Neural Computation
Year:
2010

Citing 11
Cited 2

Blind separation of sources, Part 1: an adaptive algorithm based on neuromimetic architecture

Signal Processing
Independent component analysis, a new concept?

Signal Processing - Special issue on higher order statistics
Training products of experts by minimizing contrastive divergence

Neural Computation
Estimation of Non-Normalized Statistical Models by Score Matching

The Journal of Machine Learning Research
Topographic Product Models Applied to Natural Scene Statistics

Neural Computation
A Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals

Neural Computation
Topographic Independent Component Analysis

Neural Computation
Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces

Neural Computation
Some extensions of score matching

Computational Statistics & Data Analysis
A two-layer ICA-like model estimated by score matching

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Connections Between Score Matching, Contrastive Divergence, and Pseudolikelihood for Continuous-Valued Variables

IEEE Transactions on Neural Networks

Learning invariance from natural images inspired by observations in the primary visual cortex

Neural Computation
Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

We consider a hierarchical two-layer model of natural signals in which both layers are learned from the data. Estimation is accomplished by score matching, a recently proposed estimation principle for energy-based models. If the first-layer outputs are squared and the second-layer weights are constrained to be nonnegative, the model learns responses similar to complex cells in primary visual cortex from natural images. The second layer pools a small number of features with similar orientation and frequency, but differing in spatial phase. For speech data, we obtain analogous results. The model unifies previous extensions to independent component analysis such as subspace and topographic models and provides new evidence that localized, oriented, phase-invariant features reflect the statistical properties of natural image patches.