Elements of information theory
Elements of information theory
Perception as Bayesian inference
Perception as Bayesian inference
Estimation of Non-Normalized Statistical Models by Score Matching
The Journal of Machine Learning Research
Source separation in post-nonlinear mixtures
IEEE Transactions on Signal Processing
Blind separation of mixture of independent sources through aquasi-maximum likelihood approach
IEEE Transactions on Signal Processing
Interpretation and generalization of score matching
UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
Least squares estimation without priors or supervision
Neural Computation
A connection between score matching and denoising autoencoders
Neural Computation
The Journal of Machine Learning Research
Hi-index | 0.00 |
In signal restoration by Bayesian inference, one typically uses a parametric model of the prior distribution of the signal. Here, we consider how the parameters of a prior model should be estimated from observations of uncorrupted signals. A lot of recent work has implicitly assumed that maximum likelihood estimation is the optimal estimation method. Our results imply that this is not the case. We first obtain an objective function that approximates the error occurred in signal restoration due to an imperfect prior model. Next, we show that in an important special case (small gaussian noise), the error is the same as the score-matching objective function, which was previously proposed as an alternative for likelihood based on purely computational considerations. Our analysis thus shows that score matching combines computational simplicity with statistical optimality in signal restoration, providing a viable alternative to maximum likelihood methods. We also show how the method leads to a new intuitive and geometric interpretation of structure inherent in probability distributions.