Evolutionary tuning of multiple SVM parameters

Authors:
Frauke Friedrichs;Christian Igel
Affiliations:
Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany;Institut für Neuroinformatik, Ruhr-Universität Bochum, 44780 Bochum, Germany
Venue:
Neurocomputing
Year:
2005

Citing 15
Cited 57

The nature of statistical learning theory

The nature of statistical learning theory
Making large-scale support vector machine learning practical

Advances in kernel methods
“Genotypes” for neural networks

The handbook of brain theory and neural networks
An introduction to support Vector Machines: and other kernel-based learning methods

An introduction to support Vector Machines: and other kernel-based learning methods
Soft Margins for AdaBoost

Machine Learning
Theory of evolution strategies - a tutorial

Theoretical aspects of evolutionary computing
Evolution and Optimum Seeking: The Sixth Generation

Evolution and Optimum Seeking: The Sixth Generation
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond

Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Evolution strategies –A comprehensive introduction

Natural Computing: an international journal
Choosing Multiple Parameters for Support Vector Machines

Machine Learning
Radius margin bounds for support vector machines with the RBF kernel

Neural Computation
Feature Selection for Support Vector Machines by Means of Genetic Algorithms

ICTAI '03 Proceedings of the 15th IEEE International Conference on Tools with Artificial Intelligence
Completely Derandomized Self-Adaptation in Evolution Strategies

Evolutionary Computation
Analysis of a simple evolutionary algorithm for minimization in euclidean spaces

ICALP'03 Proceedings of the 30th international conference on Automata, languages and programming
Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms

IEEE Transactions on Neural Networks

Gradient-Based Adaptation of General Gaussian Kernels

Neural Computation
2005 Special Issue: A novel approach to extracting features from motif content and protein composition for protein sequence classification

Neural Networks - Special issue on neural networks and kernel methods for structured domains
Gradient-Based Optimization of Kernel-Target Alignment for Sequence Kernels Applied to Bacterial Gene Start Detection

IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Evolving kernels for support vector machine classification

Proceedings of the 9th annual conference on Genetic and evolutionary computation
A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue

Artificial Intelligence in Medicine
Effects of SVM parameter optimization on discrimination and calibration for post-procedural PCI mortality

Journal of Biomedical Informatics
A novel LS-SVMs hyper-parameter selection based on particle swarm optimization

Neurocomputing
Uncertainty Handling in Model Selection for Support Vector Machines

Proceedings of the 10th international conference on Parallel Problem Solving from Nature: PPSN X
SVM-based segmentation and classification of remotely sensed data

International Journal of Remote Sensing
Associated evolution of a support vector machine-based classifier for pedestrian detection

Information Sciences: an International Journal
Efficient covariance matrix update for variable metric evolution strategies

Machine Learning
Kernel Trees for Support Vector Machines

IEICE - Transactions on Information and Systems
A Learning Algorithm of Boosting Kernel Discriminant Analysis for Pattern Recognition

IEICE - Transactions on Information and Systems
Support vector regression and multilayer feed forward neural networks for non-exercise prediction of VO2max

Expert Systems with Applications: An International Journal
Finding optimal model parameters by deterministic and annealed focused grid search

Neurocomputing
Learning by local kernel polarization

Neurocomputing
Short-term fault prediction based on support vector machines with parameter optimization by evolution strategy

Expert Systems with Applications: An International Journal
Weighted solution path algorithm of support vector regression based on heuristic weight-setting optimization

Neurocomputing
A novel SVR parameter selection base on bi-level programming problem

CCDC'09 Proceedings of the 21st annual international conference on Chinese control and decision conference
Evolutionary Model Type Selection for Global Surrogate Modeling

The Journal of Machine Learning Research
Simultaneous tuning of hyperparameter and parameter for support vector machines

PAKDD'07 Proceedings of the 11th Pacific-Asia conference on Advances in knowledge discovery and data mining
Experiments on kernel tree support vector machines for text categorization

PAKDD'07 Proceedings of the 11th Pacific-Asia conference on Advances in knowledge discovery and data mining
Evolutionary feature and parameter selection in support vector regression

MICAI'07 Proceedings of the artificial intelligence 6th Mexican international conference on Advances in artificial intelligence
MP-polynomial kernel for training support vector machines

CIARP'07 Proceedings of the Congress on pattern recognition 12th Iberoamerican conference on Progress in pattern recognition, image analysis and applications
Optimising multiple kernels for SVM by genetic programming

EvoCOP'08 Proceedings of the 8th European conference on Evolutionary computation in combinatorial optimization
A combination of modified particle swarm optimization algorithm and support vector machine for pattern classification

IITA'09 Proceedings of the 3rd international conference on Intelligent information technology application
An ACO-based algorithm for parameter optimization of support vector machines

Expert Systems with Applications: An International Journal
Feature selection for SVM via optimization of kernel polarization with Gaussian ARD kernels

Expert Systems with Applications: An International Journal
Covariance Matrix Self-Adaptation and Kernel Regression - Perspectives of Evolutionary Optimization in Kernel Machines

Fundamenta Informaticae - Intelligent Data Analysis in Granular Computing
Stochastic feature selection in support vector machine based instrument recognition

KI'09 Proceedings of the 32nd annual German conference on Advances in artificial intelligence
Tackling EEG signal classification with least squares support vector machines: A sensitivity analysis study

Computers in Biology and Medicine
Short term wind speed prediction based on evolutionary support vector regression algorithms

Expert Systems with Applications: An International Journal
An optimal method for prediction and adjustment on byproduct gas holder in steel industry

Expert Systems with Applications: An International Journal
Model selection for least squares support vector regressions based on small-world strategy

Expert Systems with Applications: An International Journal
Multi-objective uniform design as a SVM model selection tool for face recognition

Expert Systems with Applications: An International Journal
Evolution strategies based adaptive Lp LS-SVM

Information Sciences: an International Journal
Extended Bayesian framework for automatic tuning of kernel data-mining methods

ACS'06 Proceedings of the 6th WSEAS international conference on Applied computer science
Tuning metaheuristics: A data mining based approach for particle swarm optimization

Expert Systems with Applications: An International Journal
Learning general Gaussian kernel hyperparameters of SVMs using optimization on symmetric positive-definite matrices manifold

Pattern Recognition Letters
Combining meta-learning and search techniques to select parameters for support vector machines

Neurocomputing
Genetic programming for kernel-based learning with co-evolving subsets selection

PPSN'06 Proceedings of the 9th international conference on Parallel Problem Solving from Nature
PSO-Based hyper-parameters selection for LS-SVM classifiers

ICONIP'06 Proceedings of the 13th international conference on Neural Information Processing - Volume Part II
Evolutionary optimization of sequence kernels for detection of bacterial gene starts

ICANN'06 Proceedings of the 16th international conference on Artificial Neural Networks - Volume Part II
Swarm intelligent tuning of one-class v-SVM parameters

RSKT'06 Proceedings of the First international conference on Rough Sets and Knowledge Technology
Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process

Expert Systems with Applications: An International Journal
Statistical processes monitoring based on improved ICA and SVDD

ICIC'06 Proceedings of the 2006 international conference on Intelligent Computing - Volume Part I
Improving classification performance of Support Vector Machine by genetically optimising kernel shape and hyper-parameters

Applied Intelligence
The use of stability principle for kernel determination in relevance vector machines

ICONIP'06 Proceedings of the 13 international conference on Neural Information Processing - Volume Part I
New validation methods for improving standard and multi-parametric support vector regression training time

Expert Systems with Applications: An International Journal
An experimental study of the combination of meta-learning with particle swarm algorithms for SVM parameter selection

ICCSA'12 Proceedings of the 12th international conference on Computational Science and Its Applications - Volume Part III
Nonlinear regression in environmental sciences by support vector machines combined with evolutionary strategy

Computers & Geosciences
Automatic surrogate model type selection during the optimization of expensive black-box problems

Proceedings of the Winter Simulation Conference
An efficient multiple-kernel learning for pattern classification

Expert Systems with Applications: An International Journal
Evolutionary computation for supervised learning

Proceedings of the 15th annual conference companion on Genetic and evolutionary computation
A nested heuristic for parameter tuning in Support Vector Machines

Computers and Operations Research
A Support Vector Machine-Firefly Algorithm based forecasting model to determine malaria transmission

Neurocomputing
A fast and robust model selection algorithm for multi-input multi-output support vector machine

Neurocomputing

Quantified Score

Hi-index	0.03

Visualization

Abstract

The problem of model selection for support vector machines (SVMs) is considered. We propose an evolutionary approach to determine multiple SVM hyperparameters: The covariance matrix adaptation evolution strategy (CMA-ES) is used to determine the kernel from a parameterized kernel space and to control the regularization. Our method is applicable to optimize non-differentiable kernel functions and arbitrary model selection criteria. We demonstrate on benchmark datasets that the CMA-ES improves the results achieved by grid search already when applied to few hyperparameters. Further, we show that the CMA-ES is able to handle much more kernel parameters compared to grid-search and that tuning of the scaling and the rotation of Gaussian kernels can lead to better results in comparison to standard Gaussian kernels with a single bandwidth parameter. In particular, more flexibility of the kernel can reduce the number of support vectors.