LS-SVM hyperparameter selection with a nonparametric noise estimator

Authors:
Amaury Lendasse;Yongnan Ji;Nima Reyhani;Michel Verleysen
Affiliations:
Neural Network Research Centre, Helsinki University of Technology, Espoo, Finland;Neural Network Research Centre, Helsinki University of Technology, Espoo, Finland;Neural Network Research Centre, Helsinki University of Technology, Espoo, Finland;Université catholique de Louvain, Louvain-la-Neuve, Belgique
Venue:
ICANN'05 Proceedings of the 15th international conference on Artificial neural networks: formal models and their applications - Volume Part II
Year:
2005

Citing 2
Cited 6

Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Model selection with cross-validations and bootstraps: application to time series prediction with RBFN models

ICANN/ICONIP'03 Proceedings of the 2003 joint international conference on Artificial neural networks and neural information processing

Recursive prediction for long term time series forecasting using advanced models

Neurocomputing
On Nonparametric Residual Variance Estimation

Neural Processing Letters
Residual variance estimation in machine learning

Neurocomputing
Efficient Optimization of the Parameters of LS-SVM for Regression versus Cross-Validation Error

ICANN '09 Proceedings of the 19th International Conference on Artificial Neural Networks: Part II
Non-parametric residual variance estimation in supervised learning

IWANN'07 Proceedings of the 9th international work conference on Artificial neural networks
On incorporating seasonal information on recursive time series predictors

ICANN'07 Proceedings of the 17th international conference on Artificial neural networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

This paper presents a new method for the selection of the two hyperparameters of Least Squares Support Vector Machine (LS-SVM) approximators with Gaussian Kernels. The two hyperparameters are the width σ of the Gaussian kernels and the regularization parameter λ. For different values of σ, a Nonparametric Noise Estimator (NNE) is introduced to estimate the variance of the noise on the outputs. The NNE allows the determination of the best λ for each given σ. A Leave-one-out methodology is then applied to select the best σ. Therefore, this method transforms the double optimization problem into a single optimization one. The method is tested on 2 problems: a toy example and the Pumadyn regression Benchmark.