Neural Networks for Pattern Recognition
Neural Networks for Pattern Recognition
Training products of experts by minimizing contrastive divergence
Neural Computation
Energy-based models for sparse overcomplete representations
The Journal of Machine Learning Research
Monte Carlo Statistical Methods (Springer Texts in Statistics)
Monte Carlo Statistical Methods (Springer Texts in Statistics)
Estimation of Non-Normalized Statistical Models by Score Matching
The Journal of Machine Learning Research
Topographic Product Models Applied to Natural Scene Statistics
Neural Computation
Topographic Independent Component Analysis
Neural Computation
Training restricted Boltzmann machines using approximations to the likelihood gradient
Proceedings of the 25th international conference on Machine learning
Maximal Causes for Non-linear Component Extraction
The Journal of Machine Learning Research
Optimal approximation of signal priors
Neural Computation
Learning Features by Contrasting Natural Images with Noise
ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning
A two-layer model of natural stimuli estimated with score matching
Neural Computation
Image quality assessment: from error visibility to structural similarity
IEEE Transactions on Image Processing
Hi-index | 0.00 |
We consider the task of estimating, from observed data, a probabilistic model that is parameterized by a finite number of parameters. In particular, we are considering the situation where the model probability density function is unnormalized. That is, the model is only specified up to the partition function. The partition function normalizes a model so that it integrates to one for any choice of the parameters. However, it is often impossible to obtain it in closed form. Gibbs distributions, Markov and multi-layer networks are examples of models where analytical normalization is often impossible. Maximum likelihood estimation can then not be used without resorting to numerical approximations which are often computationally expensive. We propose here a new objective function for the estimation of both normalized and unnormalized models. The basic idea is to perform nonlinear logistic regression to discriminate between the observed data and some artificially generated noise. With this approach, the normalizing partition function can be estimated like any other parameter. We prove that the new estimation method leads to a consistent (convergent) estimator of the parameters. For large noise sample sizes, the new estimator is furthermore shown to behave like the maximum likelihood estimator. In the estimation of unnormalized models, there is a trade-off between statistical and computational performance. We show that the new method strikes a competitive trade-off in comparison to other estimation methods for unnormalized models. As an application to real data, we estimate novel two-layer models of natural image statistics with spline nonlinearities.