On the equivalence between Non-negative Matrix Factorization and Probabilistic Latent Semantic Indexing

Authors:
Chris Ding;Tao Li;Wei Peng
Affiliations:
Department of CSE, University of Texas at Arlington, Arlington, TX 76019, United States;School of Computer Science, Florida International University, Miami, FL 33199, United States;School of Computer Science, Florida International University, Miami, FL 33199, United States
Venue:
Computational Statistics & Data Analysis
Year:
2008

Citing 7
Cited 25

WebACE: a Web agent for document categorization and exploration

AGENTS '98 Proceedings of the second international conference on Autonomous agents
Document clustering based on non-negative matrix factorization

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Latent dirichlet allocation

The Journal of Machine Learning Research
Empirical and Theoretical Comparisons of Selected Criterion Functions for Document Clustering

Machine Learning
Relation between PLSA and NMF and implications

Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
A general model for clustering binary data

Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Nonnegative matrix factorization and probabilistic latent semantic indexing: equivalence, chi-square statistic, and a hybrid method

AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1

Equivalence between nonnegative tensor factorization and tensorial probabilistic latent semantic analysis

Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
NMF-based multimodal image indexing for querying by visual example

Proceedings of the ACM International Conference on Image and Video Retrieval
Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds

Information Processing and Management: an International Journal
Unsupervised clustering algorithm based on normalized Mahalanobis distances

ACACOS'10 Proceedings of the 9th WSEAS international conference on Applied computer and applied computational science
Measuring distributional similarity in context

EMNLP '10 Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing
Nonnegative tensor factorization as an alternative Csiszar---Tusnady procedure: algorithms, convergence, probabilistic interpretations and novel probabilistic tensor latent variable analysis algorithms

Data Mining and Knowledge Discovery
Community discovery using nonnegative matrix factorization

Data Mining and Knowledge Discovery
Regularized latent semantic indexing

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Kullback-Leibler divergence for nonnegative matrix factorization

ICANN'11 Proceedings of the 21th international conference on Artificial neural networks - Volume Part I
On the equivalence between nonnegative tensor factorization and tensorial probabilistic latent semantic analysis

Applied Intelligence
Extracting insights from social media with large-scale matrix approximations

IBM Journal of Research and Development
Multimodal representation, indexing, automated annotation and retrieval of image collections via non-negative matrix factorization

Neurocomputing
Quadratic nonnegative matrix factorization

Pattern Recognition
Learning evolving and emerging topics in social media: a dynamic nmf approach with temporal regularization

Proceedings of the fifth ACM international conference on Web search and data mining
A non-negative matrix factorization based approach for active dual supervision from document and word labels

EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Latent vector weighting for word meaning in context

EMNLP '11 Proceedings of the Conference on Empirical Methods in Natural Language Processing
Fast bregman divergence NMF using taylor expansion and coordinate descent

Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Exploring topic coherence over many models and many topics

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
A visual latent semantic approach for automatic analysis and interpretation of anaplastic medulloblastoma virtual slides

MICCAI'12 Proceedings of the 15th international conference on Medical Image Computing and Computer-Assisted Intervention - Volume Part I
Pairwise clustering with t-PLSI

ICANN'12 Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part II
Regularized Latent Semantic Indexing: A New Approach to Large-Scale Topic Modeling

ACM Transactions on Information Systems (TOIS)
Clustering tagged documents with labeled and unlabeled documents

Information Processing and Management: an International Journal
Combining latent factor model with location features for event-based group recommendation

Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining
Discovering different types of topics: factored topic models

IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
Adaptive multiplicative updates for quadratic nonnegative matrix factorization

Neurocomputing

Quantified Score

Hi-index	0.03

Visualization

Abstract

Non-negative Matrix Factorization (NMF) and Probabilistic Latent Semantic Indexing (PLSI) have been successfully applied to document clustering recently. In this paper, we show that PLSI and NMF (with the I-divergence objective function) optimize the same objective function, although PLSI and NMF are different algorithms as verified by experiments. This provides a theoretical basis for a new hybrid method that runs PLSI and NMF alternatively, each jumping out of the local minima of the other method successively, thus achieving a better final solution. Extensive experiments on five real-life datasets show relations between NMF and PLSI, and indicate that the hybrid method leads to significant improvements over NMF-only or PLSI-only methods. We also show that at first-order approximation, NMF is identical to the @g^2-statistic.