Divergence measures and a general framework for local variational approximation

Authors:
Kazuho Watanabe;Masato Okada;Kazushi Ikeda
Affiliations:
Graduate School of Information Science, Nara Institute of Science and Technology, 8916-5, Takayama-cho, Ikoma, Nara, 630-0192, Japan;Graduate School of Frontier Sciences, The University of Tokyo, 5-1-5, Kashiwanoha, Kashiwa, 277-8561, Japan;Graduate School of Information Science, Nara Institute of Science and Technology, 8916-5, Takayama-cho, Ikoma, Nara, 630-0192, Japan
Venue:
Neural Networks
Year:
2011

Citing 15
Cited 0

An Introduction to Variational Methods for Graphical Models

Machine Learning
Bayesian parameter estimation via variational methods

Statistics and Computing
Estimating a state-space model from point process observations

Neural Computation
A family of algorithms for approximate bayesian inference

A family of algorithms for approximate bayesian inference
Sparse bayesian learning and the relevance vector machine

The Journal of Machine Learning Research
Clustering with Bregman Divergences

The Journal of Machine Learning Research
Stochastic complexities of general mixture models in variational Bayesian learning

Neural Networks
Fast Gaussian process methods for point process intensity estimation

Proceedings of the 25th international conference on Machine learning
Bayesian Inference and Optimal Design for the Sparse Linear Model

The Journal of Machine Learning Research
Algebraic Geometry and Statistical Learning Theory

Algebraic Geometry and Statistical Learning Theory
Inferring parameters and structure of latent variable models by variational bayes

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Computing upper and lower bounds on likelihoods in intractable networks

UAI'96 Proceedings of the Twelfth international conference on Uncertainty in artificial intelligence
Bayesian hierarchical mixtures of experts

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem

IEEE Transactions on Signal Processing - Part II
Variational Gaussian process classifiers

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

The local variational method is a technique to approximate an intractable posterior distribution in Bayesian learning. This article formulates a general framework for local variational approximation and shows that its objective function is decomposable into the sum of the Kullback information and the expected Bregman divergence from the approximating posterior distribution to the Bayesian posterior distribution. Based on a geometrical argument in the space of approximating posteriors, we propose an efficient method to evaluate an upper bound of the marginal likelihood. Moreover, we demonstrate that the variational Bayesian approach for the latent variable models can be viewed as a special case of this general framework.