Communications of the ACM
A “thermal” perceptron learning rule
Neural Computation
Rigorous learning curve bounds from statistical mechanics
Machine Learning - Special issue on COLT '94
Statistical mechanical analysis of the dynamics of learning in perceptrons
Statistics and Computing
Statistical Mechanics of On-line Learning
Similarity-Based Clustering
Inference from aging information
IEEE Transactions on Neural Networks
| Hi-index | 0.00 |
We review the application of statistical mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feed-forward network learnsfrom examples generated by a time dependent teacher of the samearchitecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. Theconstructive variational method also suggests a learning algorithm. It depends, however, on some unavailable quantities, such as the present performance of the student. The construction of estimators for these quantities permits the implementation of a very effective, highly adaptive algorithm. Several other algorithms are also studied for comparison with the optimal bound and the adaptive algorithm, fordifferent types of time evolution of the rule.