2008 Special Issue: Principal whitened gradient for information geometry

Authors:
Zhirong Yang;Jorma Laaksonen
Affiliations:
Department of Information and Computer Science, Helsinki University of Technology, P.O. Box 5400, FI-02015 TKK, Espoo, Finland;Department of Information and Computer Science, Helsinki University of Technology, P.O. Box 5400, FI-02015 TKK, Espoo, Finland
Venue:
Neural Networks
Year:
2008

Citing 5
Cited 2

Natural gradient works efficiently in learning

Neural Computation
Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces

Neural Computation
Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons

Neural Computation
Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold

Neurocomputing
Discriminative components of data

IEEE Transactions on Neural Networks

Efficient Monte Carlo computation of Fisher information matrix using prior information

Computational Statistics & Data Analysis
Full Length Article: Information geometry of target tracking sensor networks

Information Fusion

Quantified Score

Hi-index	0.00

Visualization

Abstract

We propose two strategies to improve the optimization in information geometry. First, a local Euclidean embedding is identified by whitening the tangent space, which leads to an additive parameter update sequence that approximates the geodesic flow to the optimal density model. Second, removal of the minor components of gradients enhances the estimation of the Fisher information matrix and reduces the computational cost. We also prove that dimensionality reduction is necessary for learning multidimensional linear transformations. The optimization based on the principal whitened gradients demonstrates faster and more robust convergence in simulations on unsupervised learning with synthetic data and on discriminant analysis of breast cancer data.