Transformations in variational Bayesian factor analysis to speed up learning

  • Authors:
  • Jaakko Luttinen;Alexander Ilin

  • Affiliations:
  • Department of Information and Computer Science, Helsinki University of Technology TKK, P.O. Box 5400, FI-02015 TKK, Finland;Department of Information and Computer Science, Helsinki University of Technology TKK, P.O. Box 5400, FI-02015 TKK, Finland

  • Venue:
  • Neurocomputing
  • Year:
  • 2010

Quantified Score

Hi-index 0.01

Visualization

Abstract

We propose simple transformation of the hidden states in variational Bayesian factor analysis models to speed up the learning procedure. The speed-up is achieved by using proper parameterization of the posterior approximation which allows joint optimization of its individual factors, thus the transformation is theoretically justified. We derive the transformation formulae for variational Bayesian factor analysis and show experimentally that it can significantly improve the rate of convergence. The proposed transformation basically performs centering and whitening of the hidden factors taking into account the posterior uncertainties. Similar transformations can be applied to other variational Bayesian factor analysis models as well.