Exponentiated gradient versus gradient descent for linear predictors
Information and Computation
Fast training of support vector machines using sequential minimal optimization
Advances in kernel methods
Boosting as entropy projection
COLT '99 Proceedings of the twelfth annual conference on Computational learning theory
Logistic Regression, AdaBoost and Bregman Distances
Machine Learning
Efficient svm training using low-rank kernel representations
The Journal of Machine Learning Research
Efficient Margin Maximizing with Boosting
The Journal of Machine Learning Research
Greedy optimization classifiers ensemble based on diversity
Pattern Recognition
Hi-index | 0.00 |
The training of support vector machines (SVM) involves a quadratic programming problem, which is often optimized by a complicated numerical solver. In this paper, we propose a much simpler approach based on multiplicative updates. This idea was first explored in [Cristianini et al., 1999], but its convergence is sensitive to a learning rate that has to be fixed manually. Moreover, the update rule only works for the hard-margin SVM, which is known to have poor performance on noisy data. In this paper, we show that the multiplicative update of SVM can be formulated as a Bregman projection problem, and the learning rate can then be adapted automatically. Moreover, because of the connection between boosting and Bregman distance, we show that this multiplicative update for SVM can be regarded as boosting the (weighted) Parzen window classifiers. Motivated by the success of boosting, we then consider the use of an adaptive ensemble of the partially trained SVMs. Extensive experiments show that the proposed multiplicative update rule with an adaptive learning rate leads to faster and more stable convergence. Moreover, the proposed ensemble has efficient training and comparable or even better accuracy than the best-tuned soft-margin SVM.