Comparing support vector machines and feed-forward neural networks with similar parameters

Authors:
Enrique Romero;Daniel Toppo
Affiliations:
Departament de Llenguatges i Sistemes Informàtics, Universitat Politècnica de Catalunya, Barcelona, Spain;Swiss Federal Institute of Technology, I&C School of Computer and Communication Sciences, Onex, Switzerland
Venue:
IDEAL'06 Proceedings of the 7th international conference on Intelligent Data Engineering and Automated Learning
Year:
2006

Citing 8
Cited 0

The nature of statistical learning theory

The nature of statistical learning theory
Neural Networks for Pattern Recognition

Neural Networks for Pattern Recognition
Kernel Matching Pursuit

Machine Learning
A Model for Heterogeneous Neurons and Its Use in Configuring Neural Networks for Classification Problems

IWANN '97 Proceedings of the International Work-Conference on Artificial and Natural Neural Networks: Biological and Artificial Computation: From Neuroscience to Technology
Sparse bayesian learning and the relevance vector machine

The Journal of Machine Learning Research
A sequential algorithm for feed-forward neural networks with optimal coefficients and interacting frequencies

Neurocomputing
LIBSVM: A library for support vector machines

ACM Transactions on Intelligent Systems and Technology (TIST)
Constructive algorithms for structure learning in feedforward neural networks for regression problems

IEEE Transactions on Neural Networks

Quantified Score

Hi-index	0.00

Visualization

Abstract

From a computational point of view, the main differences between SVMs and FNNs are (1) how the number of elements of their respective solutions (SVM-support vectors/FNN-hidden units) is selected and (2) how the (both hidden-layer and output-layer) weights are found. Sequential FNNs, however, do not show all of these differences with respect to SVMs, since the number of hidden units is obtained as a consequence of the learning process (as for SVMs) rather than fixed a priori. In addition, there exist sequential FNNs where the hidden-layer weights are always a subset of the data, as usual for SVMs. An experimental study on several benchmark data sets, comparing several aspects of SVMs and the aforementioned sequential FNNs, is presented. The experiments were performed in the (as much as possible) same conditions for both models. Accuracies were found to be very similar. Regarding the number of support vectors, sequential FNNs constructed models with less hidden units than SVMs. In addition, all the hidden-layer weights in the FNN models were also considered as support vectors by SVMs. The computational times were lower for SVMs, with absence of numerical problems.