The Strength of Weak Learnability
Machine Learning
Machine learning: a theoretical approach
Machine learning: a theoretical approach
C4.5: programs for machine learning
C4.5: programs for machine learning
Boosting a weak learning algorithm by majority
Information and Computation
Game theory, on-line prediction and boosting
COLT '96 Proceedings of the ninth annual conference on Computational learning theory
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
Improved boosting algorithms using confidence-rated predictions
COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
On the boosting ability of top-down decision tree learning algorithms
Journal of Computer and System Sciences
Learning in Neural Networks: Theoretical Foundations
Learning in Neural Networks: Theoretical Foundations
Boosting the margin: A new explanation for the effectiveness of voting methods
ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Mutual Information Gaining Algorithm and Its Relation to PAC-Learning Algorithm
AII '94 Proceedings of the 4th International Workshop on Analogical and Inductive Inference: Algorithmic Learning Theory
DS '98 Proceedings of the First International Conference on Discovery Science
Improving Algorithms for Boosting
COLT '00 Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
Boosting Using Branching Programs
COLT '00 Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
Top-down decision tree learning as information based boosting
Theoretical Computer Science
| Hi-index | 0.00 |
Top-down algorithms such as C4.5 and CART for constructing decision trees are known to perform boosting, with the procedure of choosing classification rules at internal nodes regarded as the base learner. In this work, by introducing a notion of pseudo-entropy functions for measuring the loss of hypotheses, we give a new insight into this boosting scheme from an information-theoretic viewpoint: Whenever the base learner produces hypotheses with non-zero mutual information, the top-down algorithm reduces the conditional entropy (uncertainty) about the target function as the tree grows. Although its theoretical guarantee on its performance is worse than other popular boosting algorithms such as AdaBoost, the top-down algorithms can naturally treat multi-class classification problems. Furthermore we propose a base learner LIN that produces linear classification functions and carry out some experiments to examine the performance of the top-down algorithm with LIN as the base learner. The results show that the algorithm can sometimes perform as well as or better than AdaBoost.