Statistical Mechanics of On-line Learning

  • Authors:
  • Michael Biehl;Nestor Caticha;Peter Riegler

  • Affiliations:
  • Institute of Mathematics and Computing Science, University of Groningen, Groningen, The Netherlands 9700 AK;Instituto de Fisica, Universidade de São Paulo, São Paulo, Brazil CEP 05315-970;Fachhochschule Braunschweig/Wolfenbüttel, Fachbereich Informatik, Wolfenbüttel, Germany 38302

  • Venue:
  • Similarity-Based Clustering
  • Year:
  • 2009

Quantified Score

Hi-index 0.00

Visualization

Abstract

We introduce and discuss the application of statistical physics concepts in the context of on-line machine learning processes. The consideration of typical properties of very large systems allows to perfom averages over the randomness contained in the sequence of training data. It yields an exact mathematical description of the training dynamics in model scenarios. We present the basic concepts and results of the approach in terms of several examples, including the learning of linear separable rules, the training of multilayer neural networks, and Learning Vector Quantization.