A revised inference for correlated topic model

Authors:
Tomonari Masada;Atsuhiro Takasu
Affiliations:
Nagasaki University, Nagasaki, Japan;National Institute of Informatics, Chiyoda-ku, Tokyo, Japan
Venue:
ISNN'13 Proceedings of the 10th international conference on Advances in Neural Networks - Volume Part II
Year:
2013

Citing 4
Cited 0

On the limited memory BFGS method for large scale optimization

Mathematical Programming: Series A and B
Latent dirichlet allocation

The Journal of Machine Learning Research
On smoothing and inference for topic models

UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
Shrinkage algorithms for MMSE covariance estimation

IEEE Transactions on Signal Processing

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we provide a revised inference for correlated topic model (CTM) [3]. CTM is proposed by Blei et al. for modeling correlations among latent topics more expressively than latent Dirichlet allocation (LDA) [2] and has been attracting attention of researchers. However, we have found that the variational inference of the original paper is unstable due to almost-singularity of the covariance matrix when the number of topics is large. This means that we may be reluctant to use CTM for analyzing a large document set, which may cover a rich diversity of topics. Therefore, we revise the inference and improve its quality. First, we modify the formula for updating the covariance matrix in a manner that enables us to recover the original inference by adjusting a parameter. Second, we regularize posterior parameters for reducing a side effect caused by the formula modification. While our method is based on a heuristic intuition, an experiment conducted on large document sets showed that it worked effectively in terms of perplexity.