Model complexity control and statisticallearning theory
Natural Computing: an international journal
The Supervised Network Self-Organizing Map for Classification of Large Data Sets
Applied Intelligence
A unified method for optimizing linear image restoration filters
Signal Processing - Image and Video Coding beyond Standards
Comparison of model selection for regression
Neural Computation
The subspace information criterion for infinite dimensional hypothesis spaces
The Journal of Machine Learning Research
Motion Estimation Using Statistical Learning Theory
IEEE Transactions on Pattern Analysis and Machine Intelligence
Subspace Information Criterion for Model Selection
Neural Computation
Measuring the VC-Dimension Using Optimized Experimental Design
Neural Computation
Computational Statistics & Data Analysis
Classifier learning with a new locality regularization method
Pattern Recognition
Classifier learning with a new locality regularization method
Pattern Recognition
Linear-Time Computation of Similarity Measures for Sequential Data
The Journal of Machine Learning Research
Asymmetrical interval regression using extended ε -SVM with robust algorithm
Fuzzy Sets and Systems
Time Series Prediction Based on Generalization Bounds for Support Vector Machine
ISNN 2009 Proceedings of the 6th International Symposium on Neural Networks: Advances in Neural Networks - Part II
Radial Basis Function network learning using localized generalization error bound
Information Sciences: an International Journal
SLIT: designing complexity penalty for classification and regression trees using the SRM principle
ISNN'06 Proceedings of the Third international conference on Advances in Neural Networks - Volume Part I
Dynamic fusion method using Localized Generalization Error Model
Information Sciences: an International Journal
| Hi-index | 0.00 |
It is well known that for a given sample size there exists a model of optimal complexity corresponding to the smallest prediction (generalization) error. Hence, any method for learning from finite samples needs to have some provisions for complexity control. Existing implementations of complexity control include penalization (or regularization), weight decay (in neural networks), and various greedy procedures (aka constructive, growing, or pruning methods). There are numerous proposals for determining optimal model complexity (aka model selection) based on various (asymptotic) analytic estimates of the prediction risk and on resampling approaches. Nonasymptotic bounds on the prediction risk based on Vapnik-Chervonenkis (VC)-theory have been proposed by Vapnik. This paper describes application of VC-bounds to regression problems with the usual squared loss. An empirical study is performed for settings where the VC-bounds can be rigorously applied, i.e., linear models and penalized linear models where the VC-dimension can be accurately estimated, and the empirical risk can be reliably minimized. Empirical comparisons between model selection using VC-bounds and classical methods are performed for various noise levels, sample size, target functions and types of approximating functions. Our results demonstrate the advantages of VC-based complexity control with finite samples