The Strength of Weak Learnability
Machine Learning
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
MultiBoosting: A Technique for Combining Boosting and Wagging
Machine Learning
| Hi-index | 0.00 |
Boosting is one of the recent major developments in classification methods. The technique works by creating different versions of a classifier using an adaptive resampling procedure and then combining these classifiers using weighted voting. In this paper, several modifications of the original version of boosting, the AdaBoost algorithm introduced by Y. Freund and R.E. Schapire in 1996, will be explained. These will be shown to substantially improve the predictive power of the original version. In the first modification, weighted error estimation in AdaBoost is replaced by unweighted error estimation and this is designed to reduce the impact of observations that possess large weight. In the second modification, only a selection of base classifiers, i.e. those that contribute significantly to predictive power of the boosting model, will be included in the final model. In addition to these two modifications, we will also utilise different classification techniques as base classifiers in order to product a final boosting model. Applying these proposed modifications to three data sets from the banking industry provides results which indicate a significant and substantial improvement in predictive power over the original AdaBoost algorithm.