The Strength of Weak Learnability
Machine Learning
Hierarchical mixtures of experts and the EM algorithm
Neural Computation
The nature of statistical learning theory
The nature of statistical learning theory
Randomized algorithms
Boosting a weak learning algorithm by majority
Information and Computation
Machine Learning
On the boosting ability of top-down decision tree learning algorithms
STOC '96 Proceedings of the twenty-eighth annual ACM symposium on Theory of computing
Game theory, on-line prediction and boosting
COLT '96 Proceedings of the ninth annual conference on Computational learning theory
Improved Boosting Algorithms Using Confidence-rated Predictions
Machine Learning - The Eleventh Annual Conference on computational Learning Theory
An introduction to support Vector Machines: and other kernel-based learning methods
An introduction to support Vector Machines: and other kernel-based learning methods
Learning in Neural Networks: Theoretical Foundations
Learning in Neural Networks: Theoretical Foundations
A Theory of Learning and Generalization
A Theory of Learning and Generalization
COLT '00 Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)
Geometric Bounds for Generalization in Boosting
COLT '01/EuroCOLT '01 Proceedings of the 14th Annual Conference on Computational Learning Theory and and 5th European Conference on Computational Learning Theory
The Consistency of Greedy Algorithms for Classification
COLT '02 Proceedings of the 15th Annual Conference on Computational Learning Theory
An introduction to boosting and leveraging
Advanced lectures on machine learning
Greedy algorithms for classification—consistency, convergence rates, and adaptivity
The Journal of Machine Learning Research
Boosting with Noisy Data: Some Views from Statistical Theory
Neural Computation
| Hi-index | 0.00 |
We consider the existence of a linear weak learner for boosting algorithms. A weak learner for binary classification problems is required to achieve a weighted empirical error on the training set which is bounded from above by 1/2 − γ, γ 0, for any distribution on the data set. Moreover, in order that the weak learner be useful in terms of generalization, γ must be sufficiently far from zero. While the existence of weak learners is essential to the success of boosting algorithms, a proof of their existence based on a geometric point of view has been hitherto lacking. In this work we show that under certain natural conditions on the data set, a linear classifier is indeed a weak learner. Our results can be directly applied to generalization error bounds for boosting, leading to closed-form bounds. We also provide a procedure for dynamically determining the number of boosting iterations required to achieve low generalization error. The bounds established in this work are based on the theory of geometric discrepancy.