Implicit application of polynomial filters in a k-step Arnoldi method
SIAM Journal on Matrix Analysis and Applications
Learning a kernel matrix for nonlinear dimensionality reduction
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Smooth minimization of non-smooth functions
Mathematical Programming: Series A and B
Local Minima and Convergence in Low-Rank Semidefinite Programming
Mathematical Programming: Series A and B
Information-theoretic metric learning
Proceedings of the 24th international conference on Machine learning
A dual coordinate descent method for large-scale linear SVM
Proceedings of the 25th international conference on Machine learning
Optimization Methods & Software
Primal-dual subgradient methods for convex problems
Mathematical Programming: Series A and B - Series B - Special Issue: Nonsmooth Optimization and Applications
Distance Metric Learning for Large Margin Nearest Neighbor Classification
The Journal of Machine Learning Research
Exact Matrix Completion via Convex Optimization
Foundations of Computational Mathematics
Sparse approximate solutions to semidefinite programs
LATIN'08 Proceedings of the 8th Latin American conference on Theoretical informatics
Multi-task feature learning via efficient l2, 1-norm minimization
UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm
ACM Transactions on Algorithms (TALG)
Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach
The Journal of Machine Learning Research
Low-Rank Optimization on the Cone of Positive Semidefinite Matrices
SIAM Journal on Optimization
A Riemannian Optimization Approach for Computing Low-Rank Solutions of Lyapunov Equations
SIAM Journal on Matrix Analysis and Applications
Positive semidefinite metric learning using boosting-like algorithms
The Journal of Machine Learning Research
Low-rank quadratic semidefinite programming
Neurocomputing
| Hi-index | 0.01 |
Many machine learning tasks (e.g. metric and manifold learning problems) can be formulated as convex semidefinite programs. To enable the application of these tasks on a large-scale, scalability and computational efficiency are considered as desirable properties for a practical semidefinite programming algorithm. In this paper, we theoretically analyze a new bilateral greedy optimization (denoted BILGO) strategy in solving general semidefinite programs on large-scale datasets. As compared to existing methods, BILGO employs a bilateral search strategy during each optimization iteration. In such an iteration, the current semidefinite matrix solution is updated as a bilateral linear combination of the previous solution and a suitable rank-1 matrix, which can be efficiently computed from the leading eigenvector of the descent direction at this iteration. By optimizing for the coefficients of the bilateral combination, BILGO reduces the cost function in every iteration until the KKT conditions are fully satisfied, thus, it tends to converge to a global optimum. In fact, we prove that BILGO converges to the global optimal solution at a rate of O(1/k), where k is the iteration counter. The algorithm thus successfully combines the efficiency of conventional rank-1 update algorithms and the effectiveness of gradient descent. Moreover, BILGO can be easily extended to handle low rank constraints. To validate the effectiveness and efficiency of BILGO, we apply it to two important machine learning tasks, namely Mahalanobis metric learning and maximum variance unfolding. Extensive experimental results clearly demonstrate that BILGO can solve large-scale semidefinite programs efficiently.