2005 Special Issue: Improving dimensionality reduction with spectral gradient descent

Authors:
Roland Memisevic;Geoffrey Hinton
Affiliations:
Department of Computer Science, University of Toronto, Toronto, Ont., Canada M5S 3G4;Department of Computer Science, University of Toronto, Toronto, Ont., Canada M5S 3G4
Venue:
Neural Networks - 2005 Special issue: IJCNN 2005
Year:
2005

Citing 3
Cited 1

Nonlinear component analysis as a kernel eigenvalue problem

Neural Computation
Laplacian Eigenmaps for dimensionality reduction and data representation

Neural Computation
Principal Surfaces from Unsupervised Kernel Regression

IEEE Transactions on Pattern Analysis and Machine Intelligence

Comparison of visualization methods of genome-wide SNP profiles in childhood acute lymphoblastic leukaemia

AusDM '08 Proceedings of the 7th Australasian Data Mining Conference - Volume 87

Quantified Score

Hi-index	0.00

Visualization

Abstract

We introduce spectral gradient descent, a way of improving iterative dimensionality reduction techniques. The method uses information contained in the leading eigenvalues of a data affinity matrix to modify the steps taken during a gradient-based optimization procedure. We show that the approach is able to speed up the optimization and to help dimensionality reduction methods find better local minima of their objective functions. We also provide an interpretation of our approach in terms of the power method for finding the leading eigenvalues of a symmetric matrix and verify the usefulness of the approach in some simple experiments.