Geometric optimization methods for adaptive filtering
Geometric optimization methods for adaptive filtering
Numerical Gradient Algorithms for Eigenvalue and Singular Value Calculations
SIAM Journal on Matrix Analysis and Applications
Natural gradient works efficiently in learning
Neural Computation
The Geometry of Algorithms with Orthogonality Constraints
SIAM Journal on Matrix Analysis and Applications
Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation
International Journal of Computer Vision
Optimal Linear Representations of Images for Object Recognition
IEEE Transactions on Pattern Analysis and Machine Intelligence
A Theory for Learning by Weight Flow on Stiefel-Grassman Manifold
Neural Computation
Optimization algorithms exploiting unitary constraints
IEEE Transactions on Signal Processing
A theory for learning based on rigid bodies dynamics
IEEE Transactions on Neural Networks
Algorithms for nonnegative independent component analysis
IEEE Transactions on Neural Networks
Learning independent components on the orthogonal group of matrices by retractions
Neural Processing Letters
Neural Processing Letters
Multiplicative updates for non-negative projections
Neurocomputing
Natural Conjugate Gradient on Complex Flag Manifolds for Complex Independent Subspace Analysis
ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part I
Lie-group-type neural system learning by manifold retractions
Neural Networks
Face recognition using parzenfaces
ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Riemannian optimization method on the flag manifold for independent subspace analysis
ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
An ICA learning algorithm utilizing geodesic approach
ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
Gauss---Newton method for convex composite optimizations on Riemannian manifolds
Journal of Global Optimization
Non-convex optimization on stiefel manifold and applications to machine learning
ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part I
Entropy-based sliced inverse regression
Computational Statistics & Data Analysis
Decomposition and dictionary learning for 3D trajectories
Signal Processing
| Hi-index | 0.01 |
In this paper we extend the natural gradient method for neural networks to the case where the weight vectors are constrained to the Stiefel manifold. The proposed methods involve numerical integration techniques of the gradient flow without violating the manifold constraints. The extensions are based on geodesics. We rigorously formulate the previously proposed natural gradient and geodesics on the manifold exploiting the fact that the Stiefel manifold is a homogeneous space having a transitive action by the orthogonal group. Based on this fact, we further develop a simpler updating rule and one parameter family of its generalizations. The effectiveness of the proposed methods is validated by experiments in minor subspace analysis and independent component analysis.