2008 Special Issue: Low rank updated LS-SVM classifiers for fast variable selection

Authors:
Fabian Ojeda;Johan A. K. Suykens;Bart De Moor
Affiliations:
Department of Electrical Engineering (ESAT - SCD division), Katholieke Universiteit Leuven, B-3001 Leuven, Belgium;Department of Electrical Engineering (ESAT - SCD division), Katholieke Universiteit Leuven, B-3001 Leuven, Belgium;Department of Electrical Engineering (ESAT - SCD division), Katholieke Universiteit Leuven, B-3001 Leuven, Belgium
Venue:
Neural Networks
Year:
2008

Citing 11
Cited 6

Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
Soft Margins for AdaBoost

Machine Learning
Gene Selection for Cancer Classification using Support Vector Machines

Machine Learning
Ridge Regression Learning Algorithm in Dual Variables

ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Efficient svm training using low-rank kernel representations

The Journal of Machine Learning Research
Benchmarking Least Squares Support Vector Machine Classifiers

Machine Learning
Fast exact leave-one-out cross-validation of sparse least-squares support vector machines

Neural Networks
LS Bound based gene selection for DNA microarray data

Bioinformatics
Bounds on Error Expectation for Support Vector Machines

Neural Computation
Preventing Over-Fitting during Model Selection via Bayesian Regularisation of the Hyper-Parameters

The Journal of Machine Learning Research
Incremental training of support vector machines

IEEE Transactions on Neural Networks

Feature Selection in Survival Least Squares Support Vector Machines with Maximal Variation Constraints

IWANN '09 Proceedings of the 10th International Work-Conference on Artificial Neural Networks: Part I: Bio-Inspired Systems: Computational and Ambient Intelligence
Design of a multiple kernel learning algorithm for LS-SVM by convex programming

Neural Networks
Optimal Locality Regularized Least Squares Support Vector Machine via Alternating Optimization

Neural Processing Letters
Recursive robust least squares support vector regression based on maximum correntropy criterion

Neurocomputing
A sequential algorithm for sparse support vector classifiers

Pattern Recognition
Optimized Parameter Search for Large Datasets of the Regularization Parameter and Feature Selection for Ridge Regression

Neural Processing Letters

Quantified Score

Hi-index	0.00

Visualization

Abstract

Least squares support vector machine (LS-SVM) classifiers are a class of kernel methods whose solution follows from a set of linear equations. In this work we present low rank modifications to the LS-SVM classifiers that are useful for fast and efficient variable selection. The inclusion or removal of a candidate variable can be represented as a low rank modification to the kernel matrix (linear kernel) of the LS-SVM classifier. In this way, the LS-SVM solution can be updated rather than being recomputed, which improves the efficiency of the overall variable selection process. Relevant variables are selected according to a closed form of the leave-one-out (LOO) error estimator, which is obtained as a by-product of the low rank modifications. The proposed approach is applied to several benchmark data sets as well as two microarray data sets. When compared to other related algorithms used for variable selection, simulations applying our approach clearly show a lower computational complexity together with good stability on the generalization error.