Selective Rademacher Penalization and Reduced Error Pruning of Decision Trees
The Journal of Machine Learning Research
Approximation algorithms for minimizing empirical error by axis-parallel hyperplanes
ECML'05 Proceedings of the 16th European conference on Machine Learning
| Hi-index | 0.00 |
We consider the problem of designing a near-optimal linear decision tree to classify two given point sets B and W in $\Re^n$. A linear decision tree defines a polyhedral subdivision of space; it is a classifier if no leaf region contains points from both sets. We show hardness results for computing such a classifier with approximately optimal depth or size in polynomial time. In particular, we show that unless NP = ZPP, the depth of a classifier cannot be approximated within any constant factor, and that the total number of nodes cannot be approximated within any fixed polynomial. Our proof uses a simple connection between this problem and graph coloring and uses the result of Feige and Kilian on the inapproximability of the chromatic number. We also study the problem of designing a classifier with a single inequality that involves as few variables as possible and point out certain aspects of the difficulty of this problem.