Bayesian parameter estimation via variational methods

Authors:
Tommi S. Jaakkola;Michael I. Jordan
Affiliations:
Dept. of Elec. Eng. & Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA. tommi@ai.mit.edu;Computer Science Division and Department of Statistics, University of California, Berkeley CA. jordan@cs.berkeyley.edu
Venue:
Statistics and Computing
Year:
2000

Citing 6
Cited 57

Connectionist learning of belief networks

Artificial Intelligence
Keeping the neural networks simple by minimizing the description length of the weights

COLT '93 Proceedings of the sixth annual conference on Computational learning theory
Learning Bayesian Networks: The Combination of Knowledge and Statistical Data

Machine Learning
An introduction to variational methods for graphical models

Learning in graphical models
Probabilistic visualisation of high-dimensional binary data

Proceedings of the 1998 conference on Advances in neural information processing systems II
Mean field theory for sigmoid belief networks

Journal of Artificial Intelligence Research

Variational mixture of Bayesian independent component analyzers

Neural Computation
Tree induction vs. logistic regression: a learning-curve analysis

The Journal of Machine Learning Research
Hierarchy, priors and wavelets: structure and signal modelling using ICA

Signal Processing - Special issue on independent components analysis and beyond
The Bayesian backfitting relevance vector machine

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Lack of Consistency of Mean Field and Variational break Bayes Approximations for State Space Models

Neural Processing Letters
Compact approximations to Bayesian predictive distributions

ICML '05 Proceedings of the 22nd international conference on Machine learning
Variational Bayes for Continuous Hidden Markov Models and Its Application to Active Learning

IEEE Transactions on Pattern Analysis and Machine Intelligence
Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance

Neural Computation
On one method of non-diagonal regularization in sparse Bayesian learning

Proceedings of the 24th international conference on Machine learning
Mean-field variational approximate Bayesian inference for latent variable models

Computational Statistics & Data Analysis
Learning and approximate inference in dynamic hierarchical models

Computational Statistics & Data Analysis
A unified framework to exploit information in BCI data for continuous prediction

Neurocomputing
Weighted pseudo-metric for a fast CBIR method

Machine Graphics & Vision International Journal
Bayes Machines for binary classification

Pattern Recognition Letters
Algorithms for Sparse Linear Classifiers in the Massive Data Setting

The Journal of Machine Learning Research
Complex adaptive filtering user profile using graphical models

Information Processing and Management: an International Journal
Sharp quadratic majorization in one dimension

Computational Statistics & Data Analysis
Online Feature Selection Algorithm with Bayesian l 1 Regularization

PAKDD '09 Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining
Latent classification models for binary data

Pattern Recognition
Ranking community answers by modeling question-answer relationships via analogical reasoning

Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
A Novel Bayesian Logistic Discriminant Model with Dirichlet Distributions: An Application to Face Recognition

ICIAR '09 Proceedings of the 6th International Conference on Image Analysis and Recognition
Superresolution with compound Markov random fields via the variational EM algorithm

Neural Networks
A Bayesian Kernel logistic discriminant model: an improvement to the Kernel Fisher's discriminant

AAAI'08 Proceedings of the 23rd national conference on Artificial intelligence - Volume 3
Logistic regression models for a fast CBIR method based on feature selection

IJCAI'07 Proceedings of the 20th international joint conference on Artifical intelligence
Principal component analysis of binary data by iterated singular value decomposition

Computational Statistics & Data Analysis
A novel Bayesian logistic discriminant model: An application to face recognition

Pattern Recognition
The evidence framework applied to sparse kernel logistic regression

Neurocomputing
Multiplicative updates for L1-regularized linear and logistic regression

IDA'07 Proceedings of the 7th international conference on Intelligent data analysis
Variational GTM

IDEAL'07 Proceedings of the 8th international conference on Intelligent data engineering and automated learning
Efficient learning and feature selection in high-dimensional regression

Neural Computation
Firing rate estimation using an approximate Bayesian method

ICONIP'08 Proceedings of the 15th international conference on Advances in neuro-information processing - Volume Part I
Grid based variational approximations

Computational Statistics & Data Analysis
A note on mean-field variational approximations in Bayesian probit models

Computational Statistics & Data Analysis
Joint NDT image restoration and segmentation using gauss-Markov-Potts prior models and variational Bayesian computation

IEEE Transactions on Image Processing
A variational Bayesian methodology for hidden Markov models utilizing Student's-t mixtures

Pattern Recognition
Facial expression recognition in JAFFE dataset based on Gaussian process classification

IEEE Transactions on Neural Networks
Multitask Sparsity via Maximum Entropy Discrimination

The Journal of Machine Learning Research
Detection of hidden structures in nonstationary spike trains

Neural Computation
Bayesian belief network based broadcast sports video indexing

Multimedia Tools and Applications
User reputation in a comment rating environment

Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
Divergence measures and a general framework for local variational approximation

Neural Networks
Variational relevance vector machines

UAI'00 Proceedings of the Sixteenth conference on Uncertainty in artificial intelligence
Variational MCMC

UAI'01 Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence
A new variational Bayesian algorithm with application to human mobility pattern modeling

Statistics and Computing
Bayesian hierarchical mixtures of experts

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
A generalized mean field algorithm for variational inference in exponential families

UAI'03 Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence
Bayesian independent component analysis with prior constraints: an application in biosignal analysis

Proceedings of the First international conference on Deterministic and Statistical Methods in Machine Learning
Variational bayes estimation of mixing coefficients

Proceedings of the First international conference on Deterministic and Statistical Methods in Machine Learning
A tractable framework for estimating and combining spectral source models for audio source separation

Signal Processing
A reservoir-driven non-stationary hidden Markov model

Pattern Recognition
Bayesian inference with optimal maps

Journal of Computational Physics
Variational Bayesian inference for the Latent Position Cluster Model for network data

Computational Statistics & Data Analysis
Multi-faceted ranking of news articles using post-read actions

Proceedings of the 21st ACM international conference on Information and knowledge management
A region-centered topic model for object discovery and category-based image segmentation

Pattern Recognition
Joint modeling of imaging and genetics

IPMI'13 Proceedings of the 23rd international conference on Information Processing in Medical Imaging
Low-rank separated representation surrogates of high-dimensional stochastic functions: Application in Bayesian inference

Journal of Computational Physics
Predictive Distribution of the Dirichlet Mixture Model by Local Variational Inference

Journal of Signal Processing Systems

Quantified Score

Hi-index	0.01

Visualization

Abstract

We consider a logistic regression model with a Gaussian priordistribution over the parameters. We show that an accuratevariational transformation can be used to obtain a closed formapproximation to the posterior distribution of the parametersthereby yielding an approximate posterior predictive model. Thisapproach is readily extended to binary graphical model withcomplete observations. For graphical models with incompleteobservations we utilize an additional variational transformationand again obtain a closed form approximation to the posterior.Finally, we show that the dual of the regression problem gives alatent variable density model, the variational formulation ofwhich leads to exactly solvable EM updates.