Learning the coordinate gradients

  • Authors:
  • Yiming Ying;Qiang Wu;Colin Campbell

  • Affiliations:
  • College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, UK EX4 4EE;Department of Mathematical Sciences, Middle Tennessee State University, Murfreesboro, USA 37132;Department of Engineering Mathematics, University of Bristol, Bristol, UK

  • Venue:
  • Advances in Computational Mathematics
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

In this paper we study the problem of learning the gradient function with application to variable selection and determining variable covariation. Firstly, we propose a novel unifying framework for coordinate gradient learning from the perspective of multi-task learning. Various variable selection methods can be regarded as special instances of this framework. Secondly, we formulate the dual problems of gradient learning with general loss functions. This enables the direct application of standard optimization toolboxes to the case of gradient learning. For instance, gradient learning with SVM loss can be solved by quadratic programming (QP) routines. Thirdly, we propose a novel gradient learning formulation which can be cast as a learning the kernel matrix problem. Its relation with sparse regularization is highlighted. A semi-infinite linear programming (SILP) approach and an iterative optimization approach are proposed to efficiently solve this problem. Finally, we validate our proposed approaches on both synthetic and real datasets.