Neural networks and the bias/variance dilemma
Neural Computation
Machine Learning
Decision theoretic generalizations of the PAC model for neural net and other learning applications
Information and Computation
The nature of statistical learning theory
The nature of statistical learning theory
An experimental and theoretical comparison of model selection methods
COLT '95 Proceedings of the eighth annual conference on Computational learning theory
Characterizing the generalization performance of model selection strategies
ICML '97 Proceedings of the Fourteenth International Conference on Machine Learning
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)
A study of cross-validation and bootstrap for accuracy estimation and model selection
IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence - Volume 2
A Multistrategy Approach to Classifier Learning from Time Series
Machine Learning - Special issue on multistrategy learning
Text Classification from Labeled and Unlabeled Documents using EM
Machine Learning - Special issue on information retrieval
The Role of Occam‘s Razor in Knowledge Discovery
Data Mining and Knowledge Discovery
Model Selection for Small Sample Regression
Machine Learning
Metric-Based Methods for Adaptive Model Selection and Regularization
Machine Learning
Automatic Model Selection by Modelling the Distribution of Residuals
ECCV '02 Proceedings of the 7th European Conference on Computer Vision-Part IV
An introduction to variable and feature selection
The Journal of Machine Learning Research
Extensions to metric based model selection
The Journal of Machine Learning Research
Feature subset selection for learning preferences: a case study
ICML '04 Proceedings of the twenty-first international conference on Machine learning
Generalized Expectation Criteria for Semi-Supervised Learning with Weakly Labeled Data
The Journal of Machine Learning Research
Evaluating learning algorithms and classifiers
International Journal of Intelligent Information and Database Systems
| Hi-index | 0.00 |
We introduce a new approach to model selection that performs better than the standard complexity-penalization and hold-out error estimation techniques in many cases. The basic idea is to exploit the intrinsic metric structure of a hypothesis space, as determined by the natural distribution of unlabeled training patterns, and use this metric as a reference to detect whether the empirical error estimates derived from a small (labeled) training sample can be trusted in the region around an empirically optimal hypothesis. Using simple metric intuitions we develop new geometric strategies for detecting overfitting and performing robust yet responsive model selection in spaces of candidate functions. These new metric-based strategies dramatically outperform previous approaches in experimental studies of classical polynomial curve fitting. Moreover, the technique is simple, efficient, and can be applied to most function learning tasks. The only requirement is access to an auxiliary collection of unlabeled training data.