C4.5: programs for machine learning
C4.5: programs for machine learning
Learning in the presence of malicious errors
SIAM Journal on Computing
The nature of statistical learning theory
The nature of statistical learning theory
Lower bounds on learning decision lists and trees
Information and Computation
A decision-theoretic generalization of on-line learning and an application to boosting
Journal of Computer and System Sciences - Special issue: 26th annual ACM symposium on the theory of computing & STOC'94, May 23–25, 1994, and second annual Europe an conference on computational learning theory (EuroCOLT'95), March 13–15, 1995
A threshold of ln n for approximating set cover
Journal of the ACM (JACM)
On the boosting ability of top-down decision tree learning algorithms
Journal of Computer and System Sciences
Machine Learning
ECML '02 Proceedings of the 13th European Conference on Machine Learning
On the Power of Decision Lists
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Journal of Artificial Intelligence Research
A Real generalization of discrete AdaBoost
Artificial Intelligence
Rough approximation operators on two universes of discourse and their fuzzy extensions
Fuzzy Sets and Systems
| Hi-index | 5.23 |
One of the most popular induction scheme for supervised learning is also one of the oldest. It builds a classifier in a top-down fashion, following the minimization of a so-called index criterion. While numerous papers have reported experiments on this scheme, little has been known on its theoretical aspect until recent works on decision trees and branching programs using a powerful classification tool: boosting.In this paper, we look at this problem from a worst-case computational (rather than informational) standpoint. Our conclusions for the ranking of these indexes minimization follow almost exactly that of boosting (with matching upper and lowerbounds), and provide extensions to more classes of Boolean formulas such as decision lists, multilinear polynomials and symmetric functions. Our results also exhibit a strong worst-case for the induction scheme, as we build particularly hard samples for which the replacement of most index criteria, or the class of concept representation, even when producing the same ranking as boosting does for the indexes, makes no difference at all for the concept induced. This is clearly not a limit of previous analyses, but a consequence of the induction scheme.