Demonstrating the stability of support vector machines for classification

Authors:
I. Buciu;C. Kotropoulos;I. Pitas
Affiliations:
Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece;Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece;Department of Informatics, Aristotle University of Thessaloniki, Thessaloniki, Greece
Venue:
Signal Processing - Signal processing in UWB communications
Year:
2006

Citing 19
Cited 2

Original Contribution: Stacked generalization

Neural Networks
The nature of statistical learning theory

The nature of statistical learning theory
Bagging predictors

Machine Learning
Neural Network-Based Face Detection

IEEE Transactions on Pattern Analysis and Machine Intelligence
Example-Based Learning for View-Based Human Face Detection

IEEE Transactions on Pattern Analysis and Machine Intelligence
Bias/variance decompositions for likelihood-based estimators

Neural Computation
Boosted mixture of experts: an ensemble learning scheme

Neural Computation
An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

Machine Learning
Detecting Faces in Images: A Survey

IEEE Transactions on Pattern Analysis and Machine Intelligence
A Tutorial on Support Vector Machines for Pattern Recognition

Data Mining and Knowledge Discovery
An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants

Machine Learning
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Machine Learning
Support Vector Machine Ensemble with Bagging

SVM '02 Proceedings of the First International Workshop on Pattern Recognition with Support Vector Machines
Training Support Vector Machines: an Application to Face Detection

CVPR '97 Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97)
Multi-Modal System for Locating Heads and Faces

FG '96 Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96)
Extracting Gestural Motion Trajectories

FG '98 Proceedings of the 3rd. International Conference on Face & Gesture Recognition
Stability and generalization

The Journal of Machine Learning Research
Leave One Out Error, Stability, and Generalization of Voting Combinations of Classifiers

Machine Learning
Bias-Variance Analysis of Support Vector Machines for the Development of SVM-Based Ensemble Methods

The Journal of Machine Learning Research

On the Stability and Bias-Variance Analysis of Kernel Matrix Learning

CAI '07 Proceedings of the 20th conference of the Canadian Society for Computational Studies of Intelligence on Advances in Artificial Intelligence
Letters: On the stability and bias-variance analysis of sparse SVMs

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we deal with the stability of support vector machines (SVMs) in classification tasks. We decompose the average prediction error of SVMs into the bias and the variance terms, and we define the aggregation effect. By estimating the aforementioned terms with bootstrap smoothing techniques, we demonstrate that support vector machines are stable classifiers. To investigate the stability of the SVM several experiments were conducted. The first experiment deals with face detection. The second experiment conducted is related to the binary classification of three artificially generated data sets stemming from known distributions and an additional synthetic data set known as "Waveform". Finally, in order to support our claim on the stability of SVMs, two more binary classification experiments were carried out on the "Pime Indian Diabetes" and the "Wisconsin Breast Cancer" data sets. In general, bagging is not expected to improve the classification accuracy of SVMs.