Computing a nonnegative matrix factorization -- provably

Authors:
Sanjeev Arora;Rong Ge;Ravindran Kannan;Ankur Moitra
Affiliations:
Princeton University and Center for Computational Intractability, Princeton, NJ, USA;Princeton University and Center for Computational Intractability, Princeton, NJ, USA;Microsoft Research India, Bangalore , India;Institute for Advanced Study, Princeton, NJ, USA
Venue:
STOC '12 Proceedings of the forty-fourth annual ACM symposium on Theory of computing
Year:
2012

Citing 21
Cited 6

Solving systems of polynomial inequalities in subexponential time

Journal of Symbolic Computation
Expressing combinatorial optimization problems by linear programs

STOC '88 Proceedings of the twentieth annual ACM symposium on Theory of computing
Lower bounds for non-commutative computation

STOC '91 Proceedings of the twenty-third annual ACM symposium on Theory of computing
On the computational complexity and geometry of the first-order theory of the reals. Par I: Introduction. Preliminaries. The geometry of semi-algebraic sets. The decision problem for the existential theory of the reals

Journal of Symbolic Computation
Communication complexity and combinatorial lattice theory

Journal of Computer and System Sciences
On the combinatorial and algebraic complexity of quantifier elimination

Journal of the ACM (JACM)
Complexity and real computation

Complexity and real computation
Separable partitions

Discrete Applied Mathematics
Latent semantic indexing: a probabilistic analysis

Journal of Computer and System Sciences - Special issue on the seventeenth ACM SIGACT-SIGMOD-SIGART symposium on principles of database systems
On the complexity of K-SAT

Journal of Computer and System Sciences - Special issue on the fourteenth annual IEE conference on computational complexity
Recommendation systems: a probabilistic analysis

Journal of Computer and System Sciences - Special issue on Internet algorithms
Lectures on Discrete Geometry

Lectures on Discrete Geometry
On notions of information transfer in VLSI circuits

STOC '83 Proceedings of the fifteenth annual ACM symposium on Theory of computing
Document clustering based on non-negative matrix factorization

Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval
Latent dirichlet allocation

The Journal of Machine Learning Research
Non-negative Matrix Factorization with Sparseness Constraints

The Journal of Machine Learning Research
Using mixture models for collaborative filtering

Journal of Computer and System Sciences
On the possibility of faster SAT algorithms

SODA '10 Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms
On the Complexity of Nonnegative Matrix Factorization

SIAM Journal on Optimization
On the number of separable partitions

Journal of Combinatorial Optimization
Probabilistic latent semantic analysis

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence

Large-scale distributed non-negative sparse coding and sparse dictionary learning

Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Computational geometry column 55: new developments in nonnegative matrix factorization

ACM SIGACT News
Critical point methods and effective real algebraic geometry: new results and trends

Proceedings of the 38th international symposium on International symposium on symbolic and algebraic computation
An information complexity approach to extended formulations

Proceedings of the forty-fifth annual ACM symposium on Theory of computing
Sparse and unique nonnegative matrix factorization through data preprocessing

The Journal of Machine Learning Research
Most Tensor Problems Are NP-Hard

Journal of the ACM (JACM)

Quantified Score

Hi-index	0.00

Visualization

Abstract

The Nonnegative Matrix Factorization (NMF) problem has a rich history spanning quantum mechanics, probability theory, data analysis, polyhedral combinatorics, communication complexity, demography, chemometrics, etc. In the past decade NMF has become enormously popular in machine learning, where the factorization is computed using a variety of local search heuristics. Vavasis recently proved that this problem is NP-complete. We initiate a study of when this problem is solvable in polynomial time. Consider a nonnegative m x n matrix $M$ and a target inner-dimension r. Our results are the following: - We give a polynomial-time algorithm for exact and approximate NMF for every constant r. Indeed NMF is most interesting in applications precisely when r is small. We complement this with a hardness result, that if exact NMF can be solved in time (nm)o(r), 3-SAT has a sub-exponential time algorithm. Hence, substantial improvements to the above algorithm are unlikely. - We give an algorithm that runs in time polynomial in n, m and r under the separablity condition identified by Donoho and Stodden in 2003. The algorithm may be practical since it is simple and noise tolerant (under benign assumptions). Separability is believed to hold in many practical settings. To the best of our knowledge, this last result is the first polynomial-time algorithm that provably works under a non-trivial condition on the input matrix and we believe that this will be an interesting and important direction for future work.