Improving the training time of support vector regression algorithms through novel hyper-parameters search space reductions

Authors:
Emilio G. Ortiz-García;Sancho Salcedo-Sanz;Ángel M. Pérez-Bellido;Jose A. Portilla-Figueras
Affiliations:
Grupo de Heurísticos Modernos de Optimización y Diseño de Redes (GHEODE), Department of Signal Theory and Communications, Universidad de Alcalá, 28871 Alcalá de Henares, M ...;Grupo de Heurísticos Modernos de Optimización y Diseño de Redes (GHEODE), Department of Signal Theory and Communications, Universidad de Alcalá, 28871 Alcalá de Henares, M ...;Grupo de Heurísticos Modernos de Optimización y Diseño de Redes (GHEODE), Department of Signal Theory and Communications, Universidad de Alcalá, 28871 Alcalá de Henares, M ...;Grupo de Heurísticos Modernos de Optimización y Diseño de Redes (GHEODE), Department of Signal Theory and Communications, Universidad de Alcalá, 28871 Alcalá de Henares, M ...
Venue:
Neurocomputing
Year:
2009

Citing 7
Cited 8

Support vector machines for dynamic reconstruction of a chaotic system

Advances in kernel methods
Practical selection of SVM parameters and noise estimation for SVM regression

Neural Networks
A tutorial on support vector regression

Statistics and Computing
Model optimizing and feature selecting for support vector regression in time series forecasting

Neurocomputing
Support vector machines combined with feature selection for breast cancer diagnosis

Expert Systems with Applications: An International Journal
Support Vector Regression for the simultaneous learning of a multivariate function and its derivatives

Neurocomputing
Linear dependency between ε and the input noise in ε-support vector regression

IEEE Transactions on Neural Networks

Multi-parametric gaussian kernel function optimization for ε-SVMr using a genetic algorithm

IWANN'11 Proceedings of the 11th international conference on Artificial neural networks conference on Advances in computational intelligence - Volume Part II
Support vector regression algorithms in the forecasting of daily maximums of tropospheric ozone concentration in madrid

HAIS'10 Proceedings of the 5th international conference on Hybrid Artificial Intelligence Systems - Volume Part II
New validation methods for improving standard and multi-parametric support vector regression training time

Expert Systems with Applications: An International Journal
Efficient aerodynamic design through evolutionary programming and support vector regression algorithms

Expert Systems with Applications: An International Journal
Low-cost model selection for SVMs using local features

Engineering Applications of Artificial Intelligence
Nonlinear regression in environmental sciences by support vector machines combined with evolutionary strategy

Computers & Geosciences
Hybrid self-adaptive learning based particle swarm optimization and support vector regression model for grade estimation

Neurocomputing
Interval type-2 fuzzy weighted support vector machine learning for energy efficient biped walking

Applied Intelligence

Quantified Score

Hi-index	0.01

Visualization

Abstract

The selection of hyper-parameters in support vector machines (SVM) is a key point in the training process of these models when applied to regression problems. Unfortunately, an exact method to obtain the optimal set of SVM hyper-parameters is unknown, and search algorithms are usually applied to obtain the best possible set of hyper-parameters. In general these search algorithms are implemented as grid searches, which are time consuming, so the computational cost of the SVM training process increases considerably. This paper presents a novel study of the effect of including reductions in the range of SVM hyper-parameters, in order to reduce the SVM training time, but with the minimum possible impact in its performance. The paper presents reduction in parameter C, by considering its relation with the rest of SVM hyper-parameters (@c and @e), through an approximation of the SVM model. On the other hand, we use some characteristics of the Gaussian kernel function and a previous result in the literature to obtain novel bounds for @c and @e hyper-parameters. The search space reductions proposed are evaluated in different regression problems from UCI and StatLib databases. All the experiments carried out applying the popular LIBSVM solver have shown that our approach reduces the SVM training time, maintaining the SVM performance similar to when the complete range in SVM parameters is considered.