Upper bound for variational free energy of Bayesian networks

Authors:
Kazuho Watanabe;Motoki Shiga;Sumio Watanabe
Affiliations:
Department of Complexity Science and Engineering, The University of Tokyo, Kashiwa, Japan 277-8561;Bioinformatics Center, Institute for Chemical Research, Kyoto University, Uji, Japan 611-0011;P&I Lab., Tokyo Institute of Technology, Yokohama, Japan 226-8503
Venue:
Machine Learning
Year:
2009

Citing 15
Cited 1

Bayesian interpolation

Neural Computation
Keeping the neural networks simple by minimizing the description length of the weights

COLT '93 Proceedings of the sixth annual conference on Computational learning theory
Learning in graphical models

Learning in graphical models
Bayesian Networks and Decision Graphs

Bayesian Networks and Decision Graphs
Model Selection Criteria for Learning Belief Nets: An Empirical Comparison

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Singularities in mixture models and upper bounds of stochastic complexity

Neural Networks
Lack of Consistency of Mean Field and Variational break Bayes Approximations for State Space Models

Neural Processing Letters
Asymptotic Model Selection for Naive Bayesian Networks

The Journal of Machine Learning Research
Globally Optimal Solutions for Energy Minimization in Stereo Vision Using Reweighted Belief Propagation

ICCV '05 Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 - Volume 01
Algebraic Analysis for Nonidentifiable Learning Machines

Neural Computation
Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation

The Journal of Machine Learning Research
Inferring parameters and structure of latent variable models by variational bayes

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Stochastic complexity of bayesian networks

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Stochastic complexity for mixture of exponential families in variational bayes

ALT'05 Proceedings of the 16th international conference on Algorithmic Learning Theory
Upper bounds for variational stochastic complexities of bayesian networks

IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning

Review: learning bayesian networks: Approaches and issues

The Knowledge Engineering Review

Quantified Score

Hi-index	0.00

Visualization

Abstract

In recent years, variational Bayesian learning has been used as an approximation of Bayesian learning. In spite of the computational tractability and good generalization in many applications, its statistical properties have yet to be clarified. In this paper, we focus on variational Bayesian learning of Bayesian networks which are widely used in information processing and uncertain artificial intelligence. We derive upper bounds for asymptotic variational free energy or stochastic complexities of bipartite Bayesian networks with discrete hidden variables. Our result theoretically supports the effectiveness of variational Bayesian learning as an approximation of Bayesian learning.