Neural Computation
A practical Bayesian framework for backpropagation networks
Neural Computation
Efficient Approximations for the MarginalLikelihood of Bayesian Networks with Hidden Variables
Machine Learning - Special issue on learning with probabilistic representations
Natural gradient works efficiently in learning
Neural Computation
Bayesian Approaches to Gaussian Mixture Modeling
IEEE Transactions on Pattern Analysis and Machine Intelligence
A view of the EM algorithm that justifies incremental, sparse, and other variants
Proceedings of the NATO Advanced Study Institute on Learning in graphical models
A unifying review of linear Gaussian models
Neural Computation
Mixtures of probabilistic principal component analyzers
Neural Computation
Comparison of approximate methods for handling hyperparameters
Neural Computation
SMEM algorithm for mixture models
Proceedings of the 1998 conference on Advances in neural information processing systems II
Bayesian Learning for Neural Networks
Bayesian Learning for Neural Networks
Neural Networks for Pattern Recognition
Neural Networks for Pattern Recognition
On-line EM Algorithm for the Normalized Gaussian Network
Neural Computation
Inferring parameters and structure of latent variable models by variational bayes
UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence
Control of exploitation-exploration meta-parameter in reinforcement learning
Neural Networks - Computational models of neuromodulation
Missing Value Estimation Using Mixture of PCAs
ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Incremental Sparse Kernel Machine
ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Hierarchical Model Selection for NGnet Based on Variational Bayes Inference
ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Mining Dependence Structures from Statistical Learning Perspective
IDEAL '02 Proceedings of the Third International Conference on Intelligent Data Engineering and Automated Learning
Learning a multivariate Gaussian mixture model with the reversible jump MCMC algorithm
Statistics and Computing
Adaptive blind separation with an unknown number of sources
Neural Computation
On Bayesian principal component analysis
Computational Statistics & Data Analysis
Stochastic Complexities of Gaussian Mixtures in Variational Bayesian Approximation
The Journal of Machine Learning Research
Stochastic complexity for mixture of exponential families in generalized variational Bayes
Theoretical Computer Science
Proceedings of the 2008 ACM symposium on Applied computing
Hierarchical Bayesian Inference of Brain Activity
Neural Information Processing
Natural Conjugate Gradient in Variational Inference
Neural Information Processing
Neural Information Processing
Variational Bayesian Approach for Long-Term Relevance Feedback
PAKDD '09 Proceedings of the 13th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining
A gradient-based algorithm competitive with variational Bayesian EM for mixture of Gaussians
IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
ICANN/ICONIP'03 Proceedings of the 2003 joint international conference on Artificial neural networks and neural information processing
Prior hyperparameters in Bayesian PCA
ICANN/ICONIP'03 Proceedings of the 2003 joint international conference on Artificial neural networks and neural information processing
Generalization error of automatic relevance determination
ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
ICONIP'08 Proceedings of the 15th international conference on Advances in neuro-information processing - Volume Part I
Approximate Riemannian Conjugate Gradient Learning for Fixed-Form Variational Bayes
The Journal of Machine Learning Research
Online heterogeneous mixture modeling with marginal and copula selection
Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining
Dirichlet Process Mixtures of Generalized Linear Models
The Journal of Machine Learning Research
Stochastic complexity for mixture of exponential families in variational bayes
ALT'05 Proceedings of the 16th international conference on Algorithmic Learning Theory
Recursive inference for inverse problems using variational Bayes methodology
Proceedings of the 5th International ICST Conference on Performance Evaluation Methodologies and Tools
Variational bayesian grammar induction for natural language
ICGI'06 Proceedings of the 8th international conference on Grammatical Inference: algorithms and applications
Active learning for online bayesian matrix factorization
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Detecting changes of clustering structures using normalized maximum likelihood coding
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
EACL '12 Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics
One-class collaborative filtering with random graphs
Proceedings of the 22nd international conference on World Wide Web
Stochastic variational inference
The Journal of Machine Learning Research
| Hi-index | 0.00 |
The Bayesian framework provides a principled way of model selection. This framework estimates a probability distribution over an ensemble of models, and the prediction is done by averaging over the ensemble of models. Accordingly, the uncertainty of the models is taken into account, and complex models with more degrees of freedom are penalized. However, integration over model parameters is often intractable, and some approximation scheme is needed. Recently, a powerful approximation scheme, called the variational bayes (VB) method, has been proposed. This approach defines the free energy for a trial probability distribution, which approximates a joint posterior probability distribution over model parameters and hidden variables. The exact maximization of the free energy gives the true posterior distribution. The VB method uses factorized trial distributions. The integration over model parameters can be done analytically, and an iterative expectation-maximization-like algorithm, whose convergence is guaranteed, is derived. In this article, we derive an online version of the VB algorithm and prove its convergence by showing that it is a stochastic approximation for finding the maximum of the free energy. By combining sequential model selection procedures, the online VB method provides a fully online learning method with a model selection mechanism. In preliminary experiments using synthetic data, the online VB method was able to adapt the model structure to dynamic environments.