Inferring parameters and structure of latent variable models by variational bayes

Authors:
Hagai Attias
Affiliations:
Gatsby Unit, University College London, London, U.K.
Venue:
UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Year:
1999

Citing 17
Cited 79

Blind separation of sources, Part 1: an adaptive algorithm based on neuromimetic architecture

Signal Processing
A Bayesian Method for the Induction of Probabilistic Networks from Data

Machine Learning
Bayesian interpolation

Neural Computation
A practical Bayesian framework for backpropagation networks

Neural Computation
An information-maximization approach to blind separation and blind deconvolution

Neural Computation
Learning Bayesian Networks: The Combination of Knowledge and Statistical Data

Machine Learning
Bayesian classification (AutoClass): theory and results

Advances in knowledge discovery and data mining
Efficient Approximations for the MarginalLikelihood of Bayesian Networks with Hidden Variables

Machine Learning - Special issue on learning with probabilistic representations
Factorial Hidden Markov Models

Machine Learning - Special issue on learning with probabilistic representations
Blind source separation and deconvolution: the dynamic component analysis algorithm

Neural Computation
A view of the EM algorithm that justifies incremental, sparse, and other variants

Proceedings of the NATO Advanced Study Institute on Learning in graphical models
Choice of Basis for Laplace Approximation

Machine Learning
Learning nonlinear overcomplete representations for efficient coding

NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Independent factor analysis

Neural Computation
Mean field theory for sigmoid belief networks

Journal of Artificial Intelligence Research
The Bayesian structural EM algorithm

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Equivariant adaptive source separation

IEEE Transactions on Signal Processing

An introduction to hidden Markov models and Bayesian networks

Hidden Markov models
Bayesian model search for mixture models based on optimizing variational bounds

Neural Networks
Variational mixture of Bayesian independent component analyzers

Neural Computation
Non-linear Bayesian Image Modelling

ECCV '00 Proceedings of the 6th European Conference on Computer Vision-Part I
Incremental Sparse Kernel Machine

ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
A Probabilistic Approach to High-Resolution Sleep Analysis

ICANN '01 Proceedings of the International Conference on Artificial Neural Networks
Variational learning of clusters of undercomplete nonsymmetric independent components

The Journal of Machine Learning Research
The em algorithm for kernel matrix completion with auxiliary data

The Journal of Machine Learning Research
Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values

UAI '04 Proceedings of the 20th conference on Uncertainty in artificial intelligence
Lack of Consistency of Mean Field and Variational break Bayes Approximations for State Space Models

Neural Processing Letters
Bayesian representation learning in the cortex regulated by acetylcholine

Neural Networks
One-Shot Learning of Object Categories

IEEE Transactions on Pattern Analysis and Machine Intelligence
On the Slow Convergence of EM and VBEM in Low-Noise Linear Models

Neural Computation
Online Model Selection Based on the Variational Bayes

Neural Computation
Stochastic complexities of general mixture models in variational Bayesian learning

Neural Networks
Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories

Computer Vision and Image Understanding
Variational approximations in Bayesian model selection for finite mixture distributions

Computational Statistics & Data Analysis
Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance

Neural Computation
Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation

The Journal of Machine Learning Research
Knowledge discovery of multiple-topic document using parametric mixture model with dirichlet prior

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Stochastic complexity for mixture of exponential families in generalized variational Bayes

Theoretical Computer Science
A unified framework to exploit information in BCI data for continuous prediction

Neurocomputing
Hierarchical Bayesian Inference of Brain Activity

Neural Information Processing
Upper bound for variational free energy of Bayesian networks

Machine Learning
Analysis of Variational Bayesian Matrix Factorization

PAKDD '09 Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining
Variational Bayes for estimating the parameters of a hidden Potts model

Statistics and Computing
Letters: A novel view of the variational Bayesian clustering

Neurocomputing
Variational Bayes via propositionalized probability computation in PRISM

Annals of Mathematics and Artificial Intelligence
A note on variational Bayesian factor analysis

Neural Networks
Generalization error of linear neural networks in an empirical bayes approach

IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Latent variable model for estimation of distribution algorithm based on a probabilistic context-free grammar

IEEE Transactions on Evolutionary Computation
Distributed variational Bayesian algorithms for Gaussian mixtures in sensor networks

Signal Processing
A variational multi-view learning framework and its application to image segmentation

ICME'09 Proceedings of the 2009 IEEE international conference on Multimedia and Expo
Variational Bayesian mixture model on a subspace of exponential family distributions

IEEE Transactions on Neural Networks
Robust Bayesian mixture modelling

Neurocomputing
Multi-directional search from the primitive initial point for Gaussian mixture estimation using variational Bayes method

Neural Networks
Prior hyperparameters in Bayesian PCA

ICANN/ICONIP'03 Proceedings of the 2003 joint international conference on Artificial neural networks and neural information processing
Generalization error of automatic relevance determination

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Person name disambiguation in web pages using social network, compound words and latent topics

PAKDD'08 Proceedings of the 12th Pacific-Asia conference on Advances in knowledge discovery and data mining
Improving the robustness to outliers of mixtures of probabilistic PCAs

PAKDD'08 Proceedings of the 12th Pacific-Asia conference on Advances in knowledge discovery and data mining
Quantum annealing for variational Bayes inference

UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
Action oriented Bayesian learning of the operating space for a humanoid robot

ROBIO'09 Proceedings of the 2009 international conference on Robotics and biomimetics
Track-based self-supervised classification of dynamic obstacles

Autonomous Robots
Bayesian Learning in Sparse Graphical Factor Models via Variational Mean-Field Annealing

The Journal of Machine Learning Research
Extracting state transition dynamics from multiple spike trains using hidden markov models with correlated poisson distribution

Neural Computation
Entropy-based variational scheme for fast bayes learning of Gaussian mixtures

SSPR&SPR'10 Proceedings of the 2010 joint IAPR international conference on Structural, syntactic, and statistical pattern recognition
Learning Non-linear Multivariate Dynamics of Motion in Robotic Manipulators

International Journal of Robotics Research
Discriminative structure selection method of Gaussian Mixture Models with its application to handwritten digit recognition

Neurocomputing
Semiparametric bivariate Archimedean copulas

Computational Statistics & Data Analysis
Variational bayes for modeling score distributions

Information Retrieval
Detection of hidden structures in nonstationary spike trains

Neural Computation
Likelihood-free Bayesian estimation of multivariate quantile distributions

Computational Statistics & Data Analysis
Two design methods of hyperparameters in variational Bayes learning for Bernoulli mixtures

Neurocomputing
Online variational inference for state-space models with point-process observations

Neural Computation
Active Machine Learning for Consideration Heuristics

Marketing Science
Divergence measures and a general framework for local variational approximation

Neural Networks
Blind source separation with time series variational Bayes expectation maximization algorithm

Digital Signal Processing
Theoretical Analysis of Bayesian Matrix Factorization

The Journal of Machine Learning Research
A new variational Bayesian algorithm with application to human mobility pattern modeling

Statistics and Computing
Analytic solution of hierarchical variational bayes in linear inverse problem

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
Variational bayesian methods for audio indexing

MLMI'05 Proceedings of the Second international conference on Machine Learning for Multimodal Interaction
Stochastic complexity for mixture of exponential families in variational bayes

ALT'05 Proceedings of the 16th international conference on Algorithmic Learning Theory
Bayesian independent component analysis with prior constraints: an application in biosignal analysis

Proceedings of the First international conference on Deterministic and Statistical Methods in Machine Learning
Variational bayes estimation of mixing coefficients

Proceedings of the First international conference on Deterministic and Statistical Methods in Machine Learning
Free energy of stochastic context free grammar on variational bayes

ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
Robots that learn language: developmental approach to human-machine conversations

EELC'06 Proceedings of the Third international conference on Emergence and Evolution of Linguistic Communication: symbol Grounding and Beyond
Upper bounds for variational stochastic complexities of bayesian networks

IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning
Regularized variational bayesian learning of echo state networks with delay&sum readout

Neural Computation
Variational conditional random fields for online speaker detection and tracking

Speech Communication
Modular encoding and decoding models derived from bayesian canonical correlation analysis

Neural Computation
Global analytic solution of fully-observed variational Bayesian matrix factorization

The Journal of Machine Learning Research
Stochastic variational inference

The Journal of Machine Learning Research
QUAC: Quick unsupervised anisotropic clustering

Pattern Recognition
Learning finite Beta-Liouville mixture models via variational bayes for proportional data clustering

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
A survey on latent tree models and applications

Journal of Artificial Intelligence Research
Fast topic discovery from web search streams

Proceedings of the 23rd international conference on World wide web
Variational learning of finite Dirichlet mixture models using component splitting

Neurocomputing
A unified framework for domain independent online speaker indexing in eigen-voice space using an index tree of reference models

International Journal of Speech Technology
Structural Bayesian Linear Regression for Hidden Markov Models

Journal of Signal Processing Systems

Quantified Score

Hi-index	0.00

Visualization

Abstract

Current methods for learning graphical models with latent variables and a fixed structure estimate optimal values for the model parameters. Whereas this approach usually produces overfitting and suboptimal generalization performance, carrying out the Bayesian program of computing the full posterior distributions over the parameters remains a difficult problem. Moreover, learning the structure of models with latent variables, for which the Bayesian approach is crucial, is yet a harder problem. In this paper I present the Variational Bayes framework, which provides a solution to these problems. This approach approximates full posterior distributions over model parameters and structures, as well as latent variables, in an analytical manner without resorting to sampling methods. Unlike in the Laplace approximation, these posteriors are generally non-Gaussian and no Hessian needs to be computed. The resulting algorithm generalizes the standard Expectation Maximization algorithm, and its convergence is guaranteed. I demonstrate that this algorithm can be applied to a large class of models in several domains, including unsupervised clustering and blind source separation.