A general lower bound on the number of examples needed for learning
Information and Computation
On the sample complexity of weak learning
COLT '90 Proceedings of the third annual workshop on Computational learning theory
A statistical approach to learning and generalization in layered neural networks
COLT '89 Proceedings of the second annual workshop on Computational learning theory
Recognizing hand-printed letters and digits using backpropagation learning
Neural Computation
Bounds on the sample complexity of Bayesian learning using information theory and the VC dimension
COLT '91 Proceedings of the fourth annual workshop on Computational learning theory
Learning curves in large neural networks
COLT '91 Proceedings of the fourth annual workshop on Computational learning theory
Learnability with respect to fixed distributions
Theoretical Computer Science
How tight are the Vapnik-Chervonenkis bounds?
Neural Computation
Neural Computation
Tight bounds on transition to perfect generalization in perceptrons
Neural Computation
Decision theoretic generalizations of the PAC model for neural net and other learning applications
Information and Computation
General bounds on the number of examples needed for learning probabilistic concepts
COLT '93 Proceedings of the sixth annual conference on Computational learning theory
Measuring the VC-dimension of a learning machine
Neural Computation
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)
An experimental and theoretical comparison of model selection methods
COLT '95 Proceedings of the eighth annual conference on Computational learning theory
COLT '95 Proceedings of the eighth annual conference on Computational learning theory
On the optimal capacity of binary neural networks: rigorous combinatorial approaches
COLT '95 Proceedings of the eighth annual conference on Computational learning theory
Markov decision processes in large state spaces
COLT '95 Proceedings of the eighth annual conference on Computational learning theory
COLT '95 Proceedings of the eighth annual conference on Computational learning theory
VC dimension of an integrate-and-fire neuron model
COLT '96 Proceedings of the ninth annual conference on Computational learning theory
Towards robust model selection using estimation and approximation error bounds
COLT '96 Proceedings of the ninth annual conference on Computational learning theory
Learning curve bounds for a Markov decision process with undiscounted rewards
COLT '96 Proceedings of the ninth annual conference on Computational learning theory
An Experimental and Theoretical Comparison of Model SelectionMethods
Machine Learning - Special issue on the eighth annual conference on computational learning theory, (COLT '95)
Analytical Mean Squared Error Curves for Temporal DifferenceLearning
Machine Learning
A Guide to the Literature on Learning Probabilistic Networks from Data
IEEE Transactions on Knowledge and Data Engineering
Results in statistical discriminant analysis: a review of the former Soviet union literature
Journal of Multivariate Analysis
Where Genetic Algorithms Excel
Evolutionary Computation
Exponential or Polynomial Learning Curves? Case-Based Studies
Neural Computation
Vc dimension of an integrate-and-fire neuron model
Neural Computation
ISMIS'05 Proceedings of the 15th international conference on Foundations of Intelligent Systems
| Hi-index | 0.00 |
In this paper we introduce and investigate a mathematically rigorous theory of learning curves that is based on ideas from statistical mechanics. The advantage of our theory over the well-established Vapnik-Chervonenkis theory is that our bounds can be considerably tighter in many cases, and are also more reflective of the true behavior (functional form) of learning curves. This behavior can often exhibit dramatic properties such as phase transitions, as well as power law asymptotics not explained by the VC theory. The disadvantages of our theory are that its application requires knowledge of the input distribution, and it is limited so far to finite cardinality function classes. We illustrate our results with many concrete examples of learning curve bounds derived from our theory.