Extended SMART algorithms for non-negative matrix factorization

Authors:
Andrzej Cichocki;Shun-ichi Amari;Rafal Zdunek;Raul Kompass;Gen Hori;Zhaohui He
Affiliations:
Laboratory for Advanced Brain Signal Processing, Japan;Amari Research Unit for Mathematical Neuroscience, BSI, RIKEN, Wako-shi, Japan;Laboratory for Advanced Brain Signal Processing, Japan;Laboratory for Advanced Brain Signal Processing, Japan;Laboratory for Advanced Brain Signal Processing, Japan;Laboratory for Advanced Brain Signal Processing, Japan
Venue:
ICAISC'06 Proceedings of the 8th international conference on Artificial Intelligence and Soft Computing
Year:
2006

Citing 12
Cited 17

Information geometry of the EM and em algorithms for neural networks

Neural Networks
Robust blind source separation by beta divergence

Neural Computation
Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications

Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications
Local Non-Negative Matrix Factorization as a Visual Representation

ICDL '02 Proceedings of the 2nd International Conference on Development and Learning
Introducing a weighted non-negative matrix factorization for image classification

Pattern Recognition Letters
Divergence function, duality, and convex analysis

Neural Computation
Non-negative Matrix Factorization with Sparseness Constraints

The Journal of Machine Learning Research
Nonnegative features of spectro-temporal sounds for classification

Pattern Recognition Letters
Csiszár’s divergences for non-negative matrix factorization: family of new algorithms

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Accelerating the EMML algorithm and related iterative algorithms by rescaled block-iterative methods

IEEE Transactions on Image Processing
Choosing parameters in block-iterative or ordered subset reconstruction algorithms

IEEE Transactions on Image Processing
Gradient-based manipulation of nonparametric entropy estimates

IEEE Transactions on Neural Networks

A compact factored representation of heterogeneous subsurface scattering

ACM SIGGRAPH 2006 Papers
Nonnegative matrix factorization with constrained second-order optimization

Signal Processing
Non-negative matrix factorization with α-divergence

Pattern Recognition Letters
Nonnegative matrix factorization with quadratic programming

Neurocomputing
On α-divergence based nonnegative matrix factorization for clustering cancer gene expression data

Artificial Intelligence in Medicine
Novel Multi-layer Non-negative Tensor Factorization with Sparsity Constraints

ICANNGA '07 Proceedings of the 8th international conference on Adaptive and Natural Computing Algorithms, Part II
Regularized Alternating Least Squares Algorithms for Non-negative Matrix/Tensor Factorization

ISNN '07 Proceedings of the 4th international symposium on Neural Networks: Advances in Neural Networks, Part III
Sparse Super Symmetric Tensor Factorization

Neural Information Processing
Data Clustering with Semi-binary Nonnegative Matrix Factorization

ICAISC '08 Proceedings of the 9th international conference on Artificial Intelligence and Soft Computing
Fast and Efficient Algorithms for Nonnegative Tucker Decomposition

ISNN '08 Proceedings of the 5th international symposium on Neural Networks: Advances in Neural Networks, Part II
Fast nonnegative matrix factorization algorithms using projected gradient approaches for large-scale problems

Computational Intelligence and Neuroscience - Advances in Nonnegative Matrix and Tensor Factorization
Nonnegative matrix factorization with the itakura-saito divergence: With application to music analysis

Neural Computation
Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization

ICA'07 Proceedings of the 7th international conference on Independent component analysis and signal separation
Stability analysis of multiplicative update algorithms and application to nonnegative matrix factorization

IEEE Transactions on Neural Networks
Towards unique solutions of non-negative matrix factorization problems by a determinant criterion

Digital Signal Processing
Rank-two residue iteration method for nonnegative matrix factorization

Neurocomputing
A convergent algorithm for orthogonal nonnegative matrix factorization

Journal of Computational and Applied Mathematics

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper we derive a family of new extended SMART (Simultaneous Multiplicative Algebraic Reconstruction Technique) algorithms for Non-negative Matrix Factorization (NMF). The proposed algorithms are characterized by improved efficiency and convergence rate and can be applied for various distributions of data and additive noise. Information theory and information geometry play key roles in the derivation of new algorithms. We discuss several loss functions used in information theory which allow us to obtain generalized forms of multiplicative NMF learning adaptive algorithms. We also provide flexible and relaxed forms of the NMF algorithms to increase convergence speed and impose an additional constraint of sparsity. The scope of these results is vast since discussed generalized divergence functions include a large number of useful loss functions such as the Amari α– divergence, Relative entropy, Bose-Einstein divergence, Jensen-Shannon divergence, J-divergence, Arithmetic-Geometric (AG) Taneja divergence, etc. We applied the developed algorithms successfully to Blind (or semi blind) Source Separation (BSS) where sources may be generally statistically dependent, however are subject to additional constraints such as nonnegativity and sparsity. Moreover, we applied a novel multilayer NMF strategy which improves performance of the most proposed algorithms.