Stochastic differential equations (3rd ed.): an introduction with applications
Stochastic differential equations (3rd ed.): an introduction with applications
Bayesian Learning for Neural Networks
Bayesian Learning for Neural Networks
Training products of experts by minimizing contrastive divergence
Neural Computation
Fields of Experts: A Framework for Learning Image Priors
CVPR '05 Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2 - Volume 02
A connection between score matching and denoising autoencoders
Neural Computation
| Hi-index | 0.00 |
This letter presents an analysis of the contrastive divergence (CD) learning algorithm when applied to continuous-time linear stochastic neural networks. For this case, powerful techniques exist that allow a detailed analysis of the behavior of CD. The analysis shows that CD converges to maximum likelihood solutions only when the network structure is such that it can match the first moments of the desired distribution. Otherwise, CD can converge to solutions arbitrarily different from the log-likelihood solutions, or they can even diverge. This result suggests the need to improve our theoretical understanding of the conditions under which CD is expected to be well behaved and the conditions under which it may fail. In, addition the results point to practical ideas on how to improve the performance of CD.