The Geometry of Algorithms with Orthogonality Constraints

Authors:
Alan Edelman;Tomás A. Arias;Steven T. Smith
Affiliations:
-;-;-
Venue:
SIAM Journal on Matrix Analysis and Applications
Year:
1999

Citing 0
Cited 193

Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation

International Journal of Computer Vision
Complex-Weighted One-Unit ‘Rigid-Bodies’ Learning Rule for Independent Component Analysis

Neural Processing Letters
The Geometry of the Newton Method on Non-Compact Lie Groups

Journal of Global Optimization
Gradient Adaptive Algorithms for Contrast-Based Blind Deconvolution

Journal of VLSI Signal Processing Systems
GIPSCAL revisited. A projected gradient approach

Statistics and Computing
Kantorovich's theorem on Newton's method in Riemannian Manifolds

Journal of Complexity
Some Remarks on Numerical Methods for Second Order Differential Equations on the Orthogonal Matrix Group

ICCS '02 Proceedings of the International Conference on Computational Science-Part III
Robust Blind Source Separation Utilizing Second and Fourth Order Statistics

ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Beyond Comon's Identifiability Theorem for Independent Component Analysis

ICANN '02 Proceedings of the International Conference on Artificial Neural Networks
Optimization Criteria, Sensitivity and Robustness of Motion and Structure Estimation

ICCV '99 Proceedings of the International Workshop on Vision Algorithms: Theory and Practice
A Conjugate Gradient Method and Simulated Annealing for Blind Separation of Sources

IWANN '01 Proceedings of the 6th International Work-Conference on Artificial and Natural Neural Networks: Bio-inspired Applications of Connectionism-Part II
Symmetry Reduction of a Class of Hybrid Systems

HSCC '02 Proceedings of the 5th International Workshop on Hybrid Systems: Computation and Control
Semidefinite programming for discrete optimization and matrix completion problems

Discrete Applied Mathematics
Computation of few Lyapunov exponents by geodesic based algorithms

Future Generation Computer Systems - Special issue: Geometric numerical algorithms
Kernel independent component analysis

The Journal of Machine Learning Research
On the l1 procrustes problem

Future Generation Computer Systems - Selected papers on theoretical and computational aspects of structural dynamical systems in linear algebra and control
Statistical dynamics of on-line independent component analysis

The Journal of Machine Learning Research
Regularizing Flows for Constrained Matrix-Valued Images

Journal of Mathematical Imaging and Vision
Some optimization problems in multivariate statistics

Journal of Global Optimization
Optimal Linear Representations of Images for Object Recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence
Blind separation of sources that have spatiotemporal variance dependencies

Signal Processing - Special issue on independent components analysis and beyond
Gradient Adaptive Paraunitary Filter Banks for Spatio-Temporal Subspace Analysis and Multichannel Blind Deconvolution

Journal of VLSI Signal Processing Systems
Generalized low rank approximations of matrices

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Blind separation of positive sources by globally convergent gradient search

Neural Computation
Neural learning by geometric integration of reduced 'rigid-body' equations

Journal of Computational and Applied Mathematics
Statistical dynamics of on-line independent component analysis

The Journal of Machine Learning Research
A hierarchy of cameras for 3D photography

Computer Vision and Image Understanding - Model-based and image-based 3D scene representation for interactive visalization
A unifying model for blind separation of independent sources

Signal Processing
Symplectic methods for the approximation of the exponential map and the Newton iteration on Riemannian submanifolds

Journal of Complexity - Festschrift for the 70th birthday of Arnold Schönhage
Generalized Low Rank Approximations of Matrices

Machine Learning
A Theory for Learning by Weight Flow on Stiefel-Grassman Manifold

Neural Computation
Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons

Neural Computation
Natural Gradient Learning for Over-and Under-Complete Bases in ICA

Neural Computation
Computing electronic structures: A new multiconfiguration approach for excited states

Journal of Computational Physics
Uniqueness of the singular points of vector fields on Riemannian manifolds under the γ-condition

Journal of Complexity
Intrinsic Statistics on Riemannian Manifolds: Basic Tools for Geometric Measurements

Journal of Mathematical Imaging and Vision
Kernel Methods for Measuring Independence

The Journal of Machine Learning Research
A constrained optimization algorithm for total energy minimization in electronic structure calculations

Journal of Computational Physics
Rapid and brief communication: Multisets mixture learning-based ellipse detection

Pattern Recognition
Simultaneous modelling of the Cholesky decomposition of several covariance matrices

Journal of Multivariate Analysis
A bound on Grassmannian codes

Journal of Combinatorial Theory Series A - Special issue in honor of Jacobus H. van Lint
The use of bulk states to accelerate the band edge state calculation of a semiconductor quantum dot

Journal of Computational Physics
Inverting mappings from smooth paths through Rn to paths through Rm: A technique applied to recovering articulation from acoustics

Speech Communication
One-Bit-Matching Theorem for ICA, Convex-Concave Programming on Polyhedral Set, and Distribution Approximation for Combinatorics

Neural Computation
Learning Spectral Clustering, With Application To Speech Separation

The Journal of Machine Learning Research
Undercomplete Blind Subspace Deconvolution

The Journal of Machine Learning Research
Self-concordant functions for optimization on smooth manifolds

Journal of Global Optimization
Global registration of multiple 3D point sets via optimization-on-a-manifold

SGP '05 Proceedings of the third Eurographics symposium on Geometry processing
Overcomplete Independent Component Analysis via Linearly Constrained Minimum Variance Spatial Filtering

Journal of VLSI Signal Processing Systems
A Global Minimization Algorithm Based on a Geodesic of a Lagrangian Formulation of Newtonian Dynamics

Neural Processing Letters
Numerical methods for ordinary differential equations on matrix manifolds

Journal of Computational and Applied Mathematics
Estimation in a linear multivariate measurement error model with a change point in the data

Computational Statistics & Data Analysis
Multiplicative updates for non-negative projections

Neurocomputing
Geodesic-based and projection-based neural blind deconvolution algorithms

Signal Processing
Fast communication: Adaptive multichannel blind identification using manifold optimization

Signal Processing
2008 Special Issue: A regularized kernel CCA contrast function for ICA

Neural Networks
Two-stage optimal component analysis

Computer Vision and Image Understanding
A diversity guarantee and SNR performance for unitary limited feedback MIMO systems

EURASIP Journal on Advances in Signal Processing
Tighter bounds for random projections of manifolds

Proceedings of the twenty-fourth annual symposium on Computational geometry
Newton's method for sections on Riemannian manifolds: Generalized covariant α-theory

Journal of Complexity
Grassmann discriminant analysis: a unifying view on subspace-based learning

Proceedings of the 25th international conference on Machine learning
Hypergraph spectral learning for multi-label classification

Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining
Optimization Algorithms on Subspaces: Revisiting Missing Data Problem in Low-Rank Matrix

International Journal of Computer Vision
Affine Feature Extraction: A Generalization of the Fukunaga-Koontz Transformation

MLDM '07 Proceedings of the 5th international conference on Machine Learning and Data Mining in Pattern Recognition
Modified Modulated Hebb-Oja Learning Rule: A Method for Biologically Plausible Principal Component Analysis

Neural Information Processing
Natural Conjugate Gradient in Variational Inference

Neural Information Processing
Dimension Reduction for Mixtures of Exponential Families

ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part I
Natural Conjugate Gradient on Complex Flag Manifolds for Complex Independent Subspace Analysis

ICANN '08 Proceedings of the 18th international conference on Artificial Neural Networks, Part I
Computing several eigenpairs of Hermitian problems by conjugate gradient iterations

Journal of Computational Physics
Rolling Stiefel manifolds

International Journal of Systems Science - The Seventh Portuguese Conference on Automatic Control (Controlo'2006)
Descent methods for optimization on homogeneous manifolds

Mathematics and Computers in Simulation
A time-varying Newton algorithm for adaptive subspace tracking

Mathematics and Computers in Simulation
Orthogonal Nonnegative Matrix Factorization: Multiplicative Updates on Stiefel Manifolds

IDEAL '08 Proceedings of the 9th International Conference on Intelligent Data Engineering and Automated Learning
Affine feature extraction: A generalization of the Fukunaga-Koontz transformation

Engineering Applications of Artificial Intelligence
KSSOLV—a MATLAB toolbox for solving the Kohn-Sham equations

ACM Transactions on Mathematical Software (TOMS)
Comparing two systems: beyond common random numbers

Proceedings of the 40th Conference on Winter Simulation
Smale's point estimate theory for Newton's method on Lie groups

Journal of Complexity
Comparison of Two Main Approaches to Joint SVD

ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Hierarchical Extraction of Independent Subspaces of Unknown Dimensions

ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Soft Dimension Reduction for ICA by Joint Diagonalization on the Stiefel Manifold

ICA '09 Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation
Statistical Computing on Manifolds: From Riemannian Geometry to Computational Anatomy

Emerging Trends in Visual Computing
A geometric newton method for oja's vector field

Neural Computation
Conjugate gradient algorithm for optimization under unitary matrix constraint

Signal Processing
Nonlinear Mean Shift over Riemannian Manifolds

International Journal of Computer Vision
On the Topology and Geometry of Spaces of Affine Shapes

Journal of Mathematical Imaging and Vision
Kernel Grassmannian distances and discriminant analysis for face recognition from image sets

Pattern Recognition Letters
Intrinsic Second-Order Geometric Optimization for Robust Point Set Registration without Correspondence

EMMCVPR '09 Proceedings of the 7th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition
Projected gradient approach to the numerical solution of the SCoTLASS

Computational Statistics & Data Analysis
Spread-spectrum watermarking security

IEEE Transactions on Information Forensics and Security
Fast kernel-based independent component analysis

IEEE Transactions on Signal Processing
Geodesic prediction for limited feedback multiuser MIMO systems in temporally correlated channels

RWS'09 Proceedings of the 4th international conference on Radio and wireless symposium
Matrix completion from a few entries

ISIT'09 Proceedings of the 2009 IEEE international conference on Symposium on Information Theory - Volume 1
Stochastic orthogonal and nonorthogonal subspace basis pursuit

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Families of orthonormalization algorithms

IJCNN'09 Proceedings of the 2009 international joint conference on Neural Networks
Symplectic methods for the approximation of the exponential map and the Newton iteration on Riemannian submanifolds

Journal of Complexity - Festschrift for the 70th birthday of Arnold Schönhage
Codebook design for communication in spread and nonspread space-time block codes-based systems

ISWCS'09 Proceedings of the 6th international conference on Symposium on Wireless Communication Systems
Accurate estimation of ICA weight matrix by implicit constraint imposition using lie group

IEEE Transactions on Neural Networks
PRIMME: preconditioned iterative multimethod eigensolver—methods and software description

ACM Transactions on Mathematical Software (TOMS)
Statistical Methods and Models for Video-Based Tracking, Modeling, and Recognition

Foundations and Trends in Signal Processing
Canonical Stiefel quotient and its application to generic face recognition in illumination spaces

BTAS'09 Proceedings of the 3rd IEEE international conference on Biometrics: Theory, applications and systems
Anomaly preserving l2,∞-optimal dimensionality reduction over a Grassmann manifold

IEEE Transactions on Signal Processing
Robust nonlinear power iteration algorithm for adaptive blind separation of independent signals

Digital Signal Processing
Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold

Neurocomputing
Tools for application-driven linear dimension reduction

Neurocomputing
Geometrical methods for non-negative ICA: Manifolds, Lie groups and toral subalgebras

Neurocomputing
Relations between bundle-adjustment and epipolar-geometry-based approaches, and their applications to efficient structure from motion

Real-Time Imaging
Noncoherent MIMO communication: Grassmannian constellations and efficient detection

IEEE Transactions on Information Theory
Non-negative multilinear principal component analysis of auditory temporal modulations for music genre classification

IEEE Transactions on Audio, Speech, and Language Processing
Temporal synchronization of MIMO wireless communication in the presence of interference

IEEE Transactions on Signal Processing
Robust independent component analysis using quadratic negentropy

ICA'07 Proceedings of the 7th international conference on Independent component analysis and signal separation
Clustering limit cycle oscillators by spectral analysis of the synchronisation matrix with an additional phase sensitive rotation

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
Face recognition using parzenfaces

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks
An EM algorithm for independent component analysis using an AR-GGD source model

AI'07 Proceedings of the 20th Australian joint conference on Advances in artificial intelligence
Grassmannian predictive coding for delayed limited feedback MIMO systems

Allerton'09 Proceedings of the 47th annual Allerton conference on Communication, control, and computing
Blind sources separation algorithm based on adaptive givens rotations

ICNC'09 Proceedings of the 5th international conference on Natural computation
Optimal linear projections for enhancing desired data statistics

Statistics and Computing
Machine learning problems from optimization perspective

Journal of Global Optimization
Matrix completion from a few entries

IEEE Transactions on Information Theory
Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds

Information Processing and Management: an International Journal
Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality

Mathematics of Operations Research
Brief paper: Non-parametric methods for L2-gain estimation using iterative experiments

Automatica (Journal of IFAC)
Robust 3D object registration without explicit correspondence using geometric integration

Machine Vision and Applications - Integrated Imaging and Vision Techniques for Industrial Inspection
Matrix Completion from Noisy Entries

The Journal of Machine Learning Research
Colored subspace analysis: dimension reduction based on a signal's autocorrelation structure

IEEE Transactions on Circuits and Systems Part I: Regular Papers
A modified particle swarm optimization algorithm for the best low multilinear rank approximation of higher-order tensors

ANTS'10 Proceedings of the 7th international conference on Swarm intelligence
Aligning spatio-temporal signals on a special manifold

ECCV'10 Proceedings of the 11th European conference on Computer vision: Part V
Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank

SIAM Journal on Matrix Analysis and Applications
A Global Convergence Proof for Cyclic Jacobi Methods with Block Rotations

SIAM Journal on Matrix Analysis and Applications
Differential feedback of MIMO channel gram matrices based on geodesic curves

IEEE Transactions on Wireless Communications
Approximate Riemannian Conjugate Gradient Learning for Fixed-Form Variational Bayes

The Journal of Machine Learning Research
Lp-Nested Symmetric Distributions

The Journal of Machine Learning Research
Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach

The Journal of Machine Learning Research
Fast Algorithms for the Generalized Foley-Sammon Discriminant Analysis

SIAM Journal on Matrix Analysis and Applications
Numerical Methods for Electronic Structure Calculations of Materials

SIAM Review
Computing and Deflating Eigenvalues While Solving Multiple Right-Hand Side Linear Systems with an Application to Quantum Chromodynamics

SIAM Journal on Scientific Computing
Clifford Bundles: A Common Framework for Image, Vector Field, and Orthonormal Frame Field Regularization

SIAM Journal on Imaging Sciences
Model-Based Multiple Rigid Object Detection and Registration in Unstructured Range Data

International Journal of Computer Vision
A Riemannian Optimization Approach for Computing Low-Rank Solutions of Lyapunov Equations

SIAM Journal on Matrix Analysis and Applications
Riemannian Newton Method for the Multivariate Eigenvalue Problem

SIAM Journal on Matrix Analysis and Applications
Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors

SIAM Journal on Scientific Computing
A New Geometric Metric in the Space of Curves, and Applications to Tracking Deforming Objects by Prediction and Filtering

SIAM Journal on Imaging Sciences
DINDSCAL: direct INDSCAL

Statistics and Computing
A quasi-stochastic gradient algorithm for variance-dependent component analysis

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
One-Bit-Matching ICA theorem, convex-concave programming, and combinatorial optimization

ISNN'05 Proceedings of the Second international conference on Advances in Neural Networks - Volume Part I
Subspace estimation using projection based m-estimators over grassmann manifolds

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part I
A new discriminant subspace analysis approach for multi-class problems

Pattern Recognition
Fast kernel density independent component analysis

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Riemannian optimization method on the flag manifold for independent subspace analysis

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Relationships between the FastICA algorithm and the rayleigh quotient iteration

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
Analysis of feasible solutions of the ICA problem under the one-bit-matching condition

ICA'06 Proceedings of the 6th international conference on Independent Component Analysis and Blind Signal Separation
An ICA learning algorithm utilizing geodesic approach

ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
An implicit riemannian trust-region method for the symmetric generalized eigenproblem

ICCS'06 Proceedings of the 6th international conference on Computational Science - Volume Part I
Optimal Metric Projections for Deformable and Articulated Structure-from-Motion

International Journal of Computer Vision
On averaging in clifford groups

IWMM'04/GIAE'04 Proceedings of the 6th international conference on Computer Algebra and Geometric Algebra with Applications
Nonmonotone algorithm for minimization on closed sets with applications to minimization on Stiefel manifolds

Journal of Computational and Applied Mathematics
Adaptive model trust region methods for generalized eigenvalue problems

ICCS'05 Proceedings of the 5th international conference on Computational Science - Volume Part I
Monotonic convergence of a nonnegative ICA algorithm on stiefel manifold

ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
Trace quotient problems revisited

ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part II
Co-clustering under nonnegative matrix tri-factorization

ICONIP'11 Proceedings of the 18th international conference on Neural Information Processing - Volume Part II
A structure preserving approximation method for Hamiltonian exponential matrices

Applied Numerical Mathematics
Transfer learning to predict missing ratings via heterogeneous user feedbacks

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
Quadratic programs over the Stiefel manifold

Operations Research Letters
2011 Special Issue: A kernel-based framework to tensorial data analysis

Neural Networks
Conjugate gradient on Grassmann manifolds for robust subspace estimation

Image and Vision Computing
Advances in matrix manifolds for computer vision

Image and Vision Computing
Projection-like Retractions on Matrix Manifolds

SIAM Journal on Optimization
Weak Sharp Minima on Riemannian Manifolds

SIAM Journal on Optimization
Perturbation Theory and Optimality Conditions for the Best Multilinear Rank Approximation of a Tensor

SIAM Journal on Matrix Analysis and Applications
Note(s): On the consistency of coordinate-independent sparse estimation with BIC

Journal of Multivariate Analysis
Age invariant face verification with relative craniofacial growth model

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part VI
Multiple object tracking via prediction and filtering with a sobolev-type metric on curves

ECCV'12 Proceedings of the 12th international conference on Computer Vision - Volume Part I
Domain adaptive dictionary learning

ECCV'12 Proceedings of the 12th European conference on Computer Vision - Volume Part IV
Active subspace: Toward scalable low-rank learning

Neural Computation
Adaptive weighted orthogonal constrained algorithm for blind source separation

Digital Signal Processing
On optimizing the sum of the Rayleigh quotient and the generalized Rayleigh quotient on the unit sphere

Computational Optimization and Applications
Non-convex optimization on stiefel manifold and applications to machine learning

ICONIP'12 Proceedings of the 19th international conference on Neural Information Processing - Volume Part I
Point-process principal components analysis via geometric optimization

Neural Computation
Numerical analysis of finite dimensional approximations of Kohn---Sham models

Advances in Computational Mathematics
Sufficient dimension reduction via squared-loss mutual information estimation

Neural Computation
Sparse coding with adaptive dictionary learning for underdetermined blind speech separation

Speech Communication
Transfer learning in heterogeneous collaborative filtering domains

Artificial Intelligence
Human gesture recognition on product manifolds

The Journal of Machine Learning Research
Introduction to face recognition and evaluation of algorithm performance

Computational Statistics & Data Analysis
Trading-off among accuracy, similarity, diversity, and long-tail: a graph-based recommendation approach

Proceedings of the 7th ACM conference on Recommender systems
Linearized Alternating Direction Method with Adaptive Penalty and Warm Starts for Fast Solving Transform Invariant Low-Rank Textures

International Journal of Computer Vision
A Riemannian subgradient algorithm for economic dispatch with valve-point effect

Journal of Computational and Applied Mathematics
Geometric multiscale decompositions of dynamic low-rank matrices

Computer Aided Geometric Design
On a self-consistent-field-like iteration for maximizing the sum of the Rayleigh quotients

Journal of Computational and Applied Mathematics
Kernel analysis on Grassmann manifolds for action recognition

Pattern Recognition Letters
Approximation of the matrix exponential operator by a structure-preserving block Arnoldi-type method

Applied Numerical Mathematics
A Splitting Method for Orthogonality Constrained Problems

Journal of Scientific Computing
The geometry of fusion inspired channel design

Signal Processing
On the convergence of ICA algorithms with weighted orthogonal constraint

Digital Signal Processing

Quantified Score

Hi-index	0.16

Visualization

Abstract

In this paper we develop new Newton and conjugate gradient algorithms on the Grassmann and Stiefel manifolds. These manifolds represent the constraints that arise in such areas as the symmetric eigenvalue problem, nonlinear eigenvalue problems, electronic structures computations, and signal processing. In addition to the new algorithms, we show how the geometrical framework gives penetrating new insights allowing us to create, understand, and compare algorithms. The theory proposed here provides a taxonomy for numerical linear algebra algorithms that provide a top level mathematical view of previously unrelated algorithms. It is our hope that developers of new algorithms and perturbation theories will benefit from the theory, methods, and examples in this paper.