Nonnegative matrix factorization and probabilistic latent semantic indexing: equivalence, chi-square statistic, and a hybrid method

Authors:
Chris Ding;Tao Li;Wei Peng
Affiliations:
Lawrence Berkeley National Laboratory, Berkeley, CA;School of Computer Science, Florida International University, Miami, FL;School of Computer Science, Florida International University, Miami, FL
Venue:
AAAI'06 Proceedings of the 21st national conference on Artificial intelligence - Volume 1
Year:
2006

Citing 7
Cited 12

WebACE: a Web agent for document categorization and exploration

AGENTS '98 Proceedings of the second international conference on Autonomous agents
Document clustering based on non-negative matrix factorization

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Latent dirichlet allocation

The Journal of Machine Learning Research
Empirical and Theoretical Comparisons of Selected Criterion Functions for Document Clustering

Machine Learning
Relation between PLSA and NMF and implications

Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval
A general model for clustering binary data

Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining
Probabilistic latent semantic analysis

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence

NMF and PLSI: equivalence and a hybrid algorithm

SIGIR '06 Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval
Orthogonal nonnegative matrix t-factorizations for clustering

Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
The uncovering of hidden structures by Latent Semantic Analysis

Information Sciences: an International Journal
Unsupervised texture classification: Automatically discover and classify texture patterns

Image and Vision Computing
On the equivalence between Non-negative Matrix Factorization and Probabilistic Latent Semantic Indexing

Computational Statistics & Data Analysis
A Unified View of Matrix Factorization Models

ECML PKDD '08 Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases - Part II
Binary matrix factorization for analyzing gene expression data

Data Mining and Knowledge Discovery
Normalized dimensionality reduction using nonnegative matrix factorization

Neurocomputing
Query by document via a decomposition-based two-level retrieval approach

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval
Robust nonnegative matrix factorization using L21-norm

Proceedings of the 20th ACM international conference on Information and knowledge management
Nonnegative matrix factorizations performing object detection and localization

Applied Computational Intelligence and Soft Computing
Non-negative and sparse spectral clustering

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

Non-negative Matrix Factorization (NMF) and Probabilistic Latent Semantic Indexing (PLSI) have been successfully applied to document clustering recently. In this paper, we show that PLSI and NMF optimize the same objective function, although PLSI and NMF are different algorithms as verified by experiments. This provides a theoretical basis for a new hybrid method that runs PLSI and NMF alternatively, each jumping out of local minima of the other method successively, thus achieving better final solution. Extensive experiments on 5 real-life datasets show relations between NMF and PLSI, and indicate the hybrid method lead to significant improvements over NMF-only or PLSI-only methods. We also show that at first order approximation, NMF is identical to χ2-statistic.