Structured large margin machines: sensitive to data distributions

  • Authors:
  • Daniel S. Yeung;Defeng Wang;Wing W. Ng;Eric C. Tsang;Xizhao Wang

  • Affiliations:
  • Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong;Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong;Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong;Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong;Faculty of Mathematics and Computer Science, Hebei University, Baoding, China 071002

  • Venue:
  • Machine Learning
  • Year:
  • 2007

Quantified Score

Hi-index 0.00

Visualization

Abstract

This paper proposes a new large margin classifier--the structured large margin machine (SLMM)--that is sensitive to the structure of the data distribution. The SLMM approach incorporates the merits of "structured" learning models, such as radial basis function networks and Gaussian mixture models, with the advantages of "unstructured" large margin learning schemes, such as support vector machines and maxi-min margin machines. We derive the SLMM model from the concepts of "structured degree" and "homospace", based on an analysis of existing structured and unstructured learning models. Then, by using Ward's agglomerative hierarchical clustering on input data (or data mappings in the kernel space) to extract the underlying data structure, we formulate SLMM training as a sequential second order cone programming. Many promising features of the SLMM approach are illustrated, including its accuracy, scalability, extensibility, and noise tolerance. We also demonstrate the theoretical importance of the SLMM model by showing that it generalizes existing approaches, such as SVMs and M4s, provides novel insight into learning models, and lays a foundation for conceiving other "structured" classifiers.