Fluctuation-Dissipation Theorem and Models of Learning

Authors:
Ilya Nemenman
Affiliations:
Kavli Institute for Theoretical Physics, University of California, Santa Barbara, CA 93106, and Joint Centers for Systems Biology, Columbia University, New York, NY 10032, U.S.A.
Venue:
Neural Computation
Year:
2005

Citing 7
Cited 0

Elements of information theory

Elements of information theory
Bayesian interpolation

Neural Computation
Statistical inference, Occam's razor, and statistical mechanics on the space of probability distributions

Neural Computation
Asymmetric dynamics in optimal variance adaptation

Neural Computation
Stochastic Complexity in Statistical Inquiry Theory

Stochastic Complexity in Statistical Inquiry Theory
Bayesian computation in recurrent neural circuits

Neural Computation
Predictability, Complexity, and Learning

Neural Computation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Advances in statistical learning theory have resulted in a multitude of different designs of learning machines. But which ones are implemented by brains and other biological information processors? We analyze how various abstract Bayesian learners perform on different data and argue that it is difficult to determine which learning–theoretic computation is performed by a particular organism using just its performance in learning a stationary target (learning curve). Based on the fluctuation–dissipation relation in statistical physics, we then discuss a different experimental setup that might be able to solve the problem.