Learning Features by Contrasting Natural Images with Noise

Authors:
Michael Gutmann;Aapo Hyvärinen
Affiliations:
Dept. of Computer Science and HIIT, University of Helsinki, Finland FIN-00014;Dept. of Computer Science and HIIT, University of Helsinki, Finland FIN-00014 and Dept. of Mathematics and Statistics, University of Helsinki,
Venue:
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Year:
2009

Citing 5
Cited 1

Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
On Advances in Statistical Modeling of Natural Images

Journal of Mathematical Imaging and Vision
Energy-based models for sparse overcomplete representations

The Journal of Machine Learning Research
Estimation of Non-Normalized Statistical Models by Score Matching

The Journal of Machine Learning Research
Natural Image Statistics: A Probabilistic Approach to Early Computational Vision.

Natural Image Statistics: A Probabilistic Approach to Early Computational Vision.

Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.00

Visualization

Abstract

Modeling the statistical structure of natural images is interesting for reasons related to neuroscience as well as engineering. Currently, this modeling relies heavily on generative probabilistic models. The estimation of such models is, however, difficult, especially when they consist of multiple layers. If the goal lies only in estimating the features, i.e. in pinpointing structure in natural images, one could also estimate instead a discriminative probabilistic model where multiple layers are more easily handled. For that purpose, we propose to estimate a classifier that can tell natural images apart from reference data which has been constructed to contain some known structure of natural images. The features of the classifier then reveal the interesting structure. Here, we use a classifier with one layer of features and reference data which contains the covariance-structure of natural images. We show that the features of the classifier are similar to those which are obtained from generative probabilistic models. Furthermore, we investigate the optimal shape of the nonlinearity that is used within the classifier.