Sparse Approximate Solutions to Linear Systems
SIAM Journal on Computing
A Comparative Study on Feature Selection in Text Categorization
ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Online and batch learning of pseudo-metrics
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Learning a Mahalanobis Metric from Equivalence Constraints
The Journal of Machine Learning Research
Active Set Identification in Nonlinear Programming
SIAM Journal on Optimization
Online Passive-Aggressive Algorithms
The Journal of Machine Learning Research
A Discriminative Kernel-Based Approach to Rank Images from Text Queries
IEEE Transactions on Pattern Analysis and Machine Intelligence
Rank minimization via online learning
Proceedings of the 25th international conference on Machine learning
Introduction to Information Retrieval
Introduction to Information Retrieval
ECIR '09 Proceedings of the 31th European Conference on IR Research on Advances in Information Retrieval
Optimization Algorithms on Matrix Manifolds
Optimization Algorithms on Matrix Manifolds
Distance Metric Learning for Large Margin Nearest Neighbor Classification
The Journal of Machine Learning Research
Low-Rank Kernel Learning with Bregman Matrix Divergences
The Journal of Machine Learning Research
SIAM Journal on Matrix Analysis and Applications
Generalized Power Method for Sparse Principal Component Analysis
The Journal of Machine Learning Research
Large Scale Online Learning of Image Similarity Through Ranking
The Journal of Machine Learning Research
Matrix Completion from Noisy Entries
The Journal of Machine Learning Research
Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach
The Journal of Machine Learning Research
Low-Rank Optimization on the Cone of Positive Semidefinite Matrices
SIAM Journal on Optimization
A Riemannian Optimization Approach for Computing Low-Rank Solutions of Lyapunov Equations
SIAM Journal on Matrix Analysis and Applications
SIAM Journal on Matrix Analysis and Applications
Hierarchical semantic indexing for large scale image retrieval
CVPR '11 Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
WSABIE: scaling up to large vocabulary image annotation
IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
Hi-index | 0.00 |
When learning models that are represented in matrix forms, enforcing a low-rank constraint can dramatically improve the memory and run time complexity, while providing a natural regularization of the model. However, naive approaches to minimizing functions over the set of low-rank matrices are either prohibitively time consuming (repeated singular value decomposition of the matrix) or numerically unstable (optimizing a factored representation of the low-rank matrix). We build on recent advances in optimization over manifolds, and describe an iterative online learning procedure, consisting of a gradient step, followed by a second-order retraction back to the manifold. While the ideal retraction is costly to compute, and so is the projection operator that approximates it, we describe another retraction that can be computed efficiently. It has run time and memory complexity of O((n+m)k) for a rank-k matrix of dimension m×n, when using an online procedure with rank-one gradients. We use this algorithm, LORETA, to learn a matrix-form similarity measure over pairs of documents represented as high dimensional vectors. LORETA improves the mean average precision over a passive-aggressive approach in a factorized model, and also improves over a full model trained on pre-selected features using the same memory requirements. We further adapt LORETA to learn positive semi-definite low-rank matrices, providing an online algorithm for low-rank metric learning. LORETA also shows consistent improvement over standard weakly supervised methods in a large (1600 classes and 1 million images, using ImageNet) multilabel image classification task.