Induction of one-level decision trees
ML92 Proceedings of the ninth international workshop on Machine learning
Decision theoretic generalizations of the PAC model for neural net and other learning applications
Information and Computation
Rigorous learning curve bounds from statistical mechanics
Machine Learning - Special issue on COLT '94
Learning one-variable pattern languages in linear average time
COLT' 98 Proceedings of the eleventh annual conference on Computational learning theory
Estimating the expected error of empirical minimizers for model selection
AAAI '98/IAAI '98 Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence
Nonparametric Regularization of Decision Trees
ECML '00 Proceedings of the 11th European Conference on Machine Learning
A Process-Oriented Heuristic for Model Selection
ICML '98 Proceedings of the Fifteenth International Conference on Machine Learning
Expected Error Analysis for Model Selection
ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
Generalized Average-Case Analyses of the Nearest Neighbor Algorithm
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Predicting the Generalization Performance of Cross Validatory Model Selection Criteria
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
Generalization Error of Limear Neural Networks in Unidentifiable Cases
ALT '99 Proceedings of the 10th International Conference on Algorithmic Learning Theory
Computable Shell Decomposition Bounds
COLT '00 Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
Tractable Average-Case Analysis of Naive Bayesian Classifiers
ICML '99 Proceedings of the Sixteenth International Conference on Machine Learning
An average-case analysis of the k-nearest neighbor classifier for noisy domains
IJCAI'97 Proceedings of the 15th international joint conference on Artifical intelligence - Volume 1
Process-oriented estimation of generalization error
IJCAI'99 Proceedings of the 16th international joint conference on Artificial intelligence - Volume 2
| Hi-index | 0.00 |
We conduct an average-case analysis of the generalization error rate of classification algorithms with finite model classes. Unlike worst-case approaches, we do not rely on bounds that hold for all possible learning problems. Instead, we study the behavior of a learning algorithm for a given problem, taking properties of the problem and the learner into account. The solution depends only on known quantities (e.g., the sample size), and the histogram of error rates in the model class which we determine for the case that the sought target is a randomly drawn Boolean function. We then discuss how the error histogram can be estimated from a given sample and thus show how the analysis can be applied approximately in the more realistic scenario that the target is unknown. Experiments show that our analysis can predict the behavior of decision tree algorithms fairly accurately even if the error histogram is estimated from a sample.