Perturbation-Based Eigenvector Updates for On-Line Principal Components Analysis and Canonical Correlation Analysis

Authors:
Anant Hegde;Jose C. Principe;Deniz Erdogmus;Umut Ozertem;Yadunandana N. Rao;Hemanth Peddaneni
Affiliations:
CNEL, ECE Department, University of Florida, Gainesville, USA;CNEL, ECE Department, University of Florida, Gainesville, USA;CSEE Department, Oregon Health & Science University, Portland, USA;CSEE Department, Oregon Health & Science University, Portland, USA;Motorola, Plantation, USA;Neurotronics, Inc, Gainesville, USA
Venue:
Journal of VLSI Signal Processing Systems
Year:
2006

Citing 5
Cited 1

Original Contribution: Least mean square error reconstruction principle for self-organizing neural-nets

Neural Networks
Adaptive filter theory (3rd ed.)

Adaptive filter theory (3rd ed.)
Simultaneous principal-component extraction with application to adaptive blind multiuser detection

EURASIP Journal on Applied Signal Processing
Projection approximation subspace tracking

IEEE Transactions on Signal Processing
Artificial neural networks for feature extraction and multivariate data projection

IEEE Transactions on Neural Networks

Locally Defined Principal Curves and Surfaces

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.01

Visualization

Abstract

Principal components analysis is an important and well-studied subject in statistics and signal processing. Several algorithms for solving this problem exist, and could be mostly grouped into one of the following three approaches: adaptation based on Hebbian updates and deflation, optimization of a second order statistical criterion (like reconstruction error or output variance), and fixed point update rules with deflation. In this study, we propose an alternate approach that avoids deflation and gradient-search techniques. The proposed method is an on-line procedure based on recursively updating the eigenvector and eigenvalue matrices with every new sample such that the estimates approximately track their true values as would be calculated analytically from the current sample estimate of the data covariance matrix. The perturbation technique is theoretically shown to be applicable for recursive canonical correlation analysis, as well. The performance of this algorithm is compared with that of a structurally similar matrix perturbation-based method and also with a few other traditional methods like Sanger's rule and APEX.