Efficient kernel learning from side information using ADMM

  • Authors:
  • En-Liang Hu;James T. Kwok

  • Affiliations:
  • Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong and Department of Mathematics, Yunnan Normal University, Yunnan, China;Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong

  • Venue:
  • IJCAI'13 Proceedings of the Twenty-Third international joint conference on Artificial Intelligence
  • Year:
  • 2013

Quantified Score

Hi-index 0.00

Visualization

Abstract

Side information is highly useful in the learning of a nonparametric kernel matrix. However, this often leads to an expensive semidefinite program (SDP). In recent years, a number of dedicated solvers have been proposed. Though much better than off-the-shelf SDP solvers, they still cannot scale to large data sets. In this paper, we propose a novel solver based on the alternating direction method of multipliers (ADMM). The key idea is to use a low-rank decomposition of the kernel matrix K = VTU, with the constraint that V = U. The resultant optimization problem, though non-convex, has favorable convergence properties and can be efficiently solved without requiring eigen-decomposition in each iteration. Experimental results on a number of real-world data sets demonstrate that the proposed method is as accurate as directly solving the SDP, but can be one to two orders of magnitude faster.