Steering time-dependent estimation of posteriors with hyperparameter indexing in bayesian topic models

Authors:
Atsuhiro Takasu;Yuichiro Shibata;Kiyoshi Oguri
Affiliations:
National Institute of Informatics, Chiyoda-ku, Tokyo, Japan;Nagasaki University, Nagasaki-shi, Nagasaki, Japan;Nagasaki University, Nagasaki-shi, Nagasaki, Japan
Venue:
PAKDD'11 Proceedings of the 15th Pacific-Asia conference on Advances in knowledge discovery and data mining - Volume Part I
Year:
2011

Citing 9
Cited 0

Probabilistic latent semantic indexing

Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval
Latent dirichlet allocation

The Journal of Machine Learning Research
Dynamic topic models

ICML '06 Proceedings of the 23rd international conference on Machine learning
Topics over time: a non-Markov continuous-time model of topical trends

Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
Multiscale topic tomography

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
The dynamic hierarchical Dirichlet process

Proceedings of the 25th international conference on Machine learning
On smoothing and inference for topic models

UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
Hierarchical Bayesian Modeling of Topics in Time-Stamped Documents

IEEE Transactions on Pattern Analysis and Machine Intelligence
Online multiscale dynamic topic models

Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper provides a new approach to topical trend analysis. Our aim is to improve the generalization power of latent Dirichlet allocation (LDA) by using document timestamps. Many previous works model topical trends by making latent topic distributions timedependent. We propose a straightforward approach by preparing a different word multinomial distribution for each time point. Since this approach increases the number of parameters, overfitting becomes a critical issue. Our contribution to this issue is two-fold. First, we propose an effective way of defining Dirichlet priors over the word multinomials. Second, we propose a special scheduling of variational Bayesian (VB) inference. Comprehensive experiments with six datasets prove that our approach can improve LDA and also Topics over Time, a well-known variant of LDA, in terms of test data perplexity in the framework of VB inference.