A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
COLT '99 Proceedings of the twelfth annual conference on Computational learning theory
PAC-Bayesian Stochastic Model Selection
Machine Learning
An Improved Predictive Accuracy Bound for Averaging Classifiers
ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Pac-bayesian generalisation error bounds for gaussian process classification
The Journal of Machine Learning Research
Tutorial on Practical Prediction Theory for Classification
The Journal of Machine Learning Research
PAC-Bayesian learning of linear classifiers
ICML '09 Proceedings of the 26th Annual International Conference on Machine Learning
| Hi-index | 0.00 |
We propose algorithms for producing weighted majority votes that learn by probing the empirical risk of a randomized (uniformly weighted) majority vote--instead of probing the zero-one loss, at some margin level, of the deterministic weighted majority vote as it is often proposed. The learning algorithms minimize a risk bound which is convex in the weights. Our numerical results indicate that learners producing a weighted majority vote based on the empirical risk of the randomized majority vote at some finite margin have no significant advantage over learners that achieve this same task based on the empirical risk at zero margin. We also find that it is sufficient for learners to minimize only the empirical risk of the randomized majority vote at a fixed number of voters without considering explicitly the entropy of the distribution of voters. Finally, our extensive numerical results indicate that the proposed learning algorithms are producing weighted majority votes that generally compare favorably to those produced by AdaBoost.