Distance metric learning by minimal distance maximization

  • Authors:
  • Yaoliang Yu;Jiayan Jiang;Liming Zhang

  • Affiliations:
  • Department of Computing Science, University of Alberta, Edmonton, AB, Canada T6G 2E8;Department of Electronic Engineering, Fudan University, Shanghai 200433, PR China;Department of Electronic Engineering, Fudan University, Shanghai 200433, PR China

  • Venue:
  • Pattern Recognition
  • Year:
  • 2011

Quantified Score

Hi-index 0.01

Visualization

Abstract

Classic linear dimensionality reduction (LDR) methods, such as principal component analysis (PCA) and linear discriminant analysis (LDA), are known not to be robust against outliers. Following a systematic analysis of the multi-class LDR problem in a unified framework, we propose a new algorithm, called minimal distance maximization (MDM), to address the non-robustness issue. The principle behind MDM is to maximize the minimal between-class distance in the output space. MDM is formulated as a semi-definite program (SDP), and its dual problem reveals a close connection to ''weighted'' LDR methods. A soft version of MDM, in which LDA is subsumed as a special case, is also developed to deal with overlapping centroids. Finally, we drop the homoscedastic Gaussian assumption made in MDM by extending it in a non-parametric way, along with a gradient-based convex approximation algorithm to significantly reduce the complexity of the original SDP. The effectiveness of our proposed methods are validated on two UCI datasets and two face datasets.