Improving classification performance of Support Vector Machine by genetically optimising kernel shape and hyper-parameters

  • Authors:
  • Laura Dioşan;Alexandrina Rogozan;Jean-Pierre Pecuchet

  • Affiliations:
  • Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes, EA 4108, Institut National des Sciences Appliquées, Rouen, France and Faculty of Mathematics and Computer Scie ...;Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes, EA 4108, Institut National des Sciences Appliquées, Rouen, France;Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes, EA 4108, Institut National des Sciences Appliquées, Rouen, France

  • Venue:
  • Applied Intelligence
  • Year:
  • 2012

Quantified Score

Hi-index 0.00

Visualization

Abstract

Support Vector Machines (SVMs) deliver state-of-the-art performance in real-world applications and are now established as one of the standard tools for machine learning and data mining. A key problem of these methods is how to choose an optimal kernel and how to optimise its parameters. The real-world applications have also emphasised the need to consider a combination of kernels--a multiple kernel--in order to boost the classification accuracy by adapting the kernel to the characteristics of heterogeneous data. This combination could be linear or non-linear, weighted or un-weighted. Several approaches have been already proposed to find a linear weighted kernel combination and to optimise its parameters together with the SVM parameters, but no approach has tried to optimise a non-linear weighted combination. Therefore, our goal is to automatically generate and adapt a kernel combination (linear or non-linear, weighted or un-weighted, according to the data) and to optimise both the kernel parameters and SVM parameters by evolutionary means in a unified framework. We will denote our combination as a kernel of kernels (KoK). Numerical experiments show that the SVM algorithm, involving the evolutionary kernel of kernels (eKoK) we propose, performs better than well-known classic kernels whose parameters were optimised and a state of the art convex linear and an evolutionary linear, respectively, kernel combinations. These results emphasise the fact that the SVM algorithm could require a non-linear weighted combination of kernels.