Kolmogorov's Structure Functions with an Application to the Foundations of Model Selection

  • Authors:
  • Nikolai K. Vereshchagin;Paul M. B. Vitányi

  • Affiliations:
  • -;-

  • Venue:
  • FOCS '02 Proceedings of the 43rd Symposium on Foundations of Computer Science
  • Year:
  • 2002

Quantified Score

Hi-index 0.00

Visualization

Abstract

In 1974 Kolmogorov proposed a non-probabilistic approach to statistics, an individual combinatorial relation between the data and its model. We vindicate, for the first time, the rightness of the original "structure function", proposed by Kolmogorov: minimizing the data-to-model code length (finding the ML estimator or MDL estimator), in a class of contemplated models of prescribed maximal (Kolmogorov) complexity, always results in a model of best fit (expressed as minimal randomness deficiency). We show that both the structure function and the minimum randomness deficiency function can assume all shapes over their full domain (improving an old result of L.A. Levin and both an old and a recent one of V.V. Vyugin). We determine the (un)computability properties of the various functions and "algorithmic sufficient statistic."