Applications of Kolmogorov Complexity and Universal Codes to Nonparametric Estimation of Characteristics of Time Series

  • Authors:
  • Boris Ryabko

  • Affiliations:
  • Siberian State University of Telecommunications and Informatics, Institute of Computational Technologies of Siberian Branch of Russian Academy of Science Kirov Street, 86, 630102, Novosibirsk, Rus ...

  • Venue:
  • Fundamenta Informaticae
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

We consider finite-alphabet and real-valued time series and thefollowing four problems: i) estimation of the (limiting)probability P(x_0 ... x_s) for every s and each sequence x_0 ...x_s of letters from the process alphabet (or estimation of thedensity p(x_0, ..., x_s) for real-valued time series), ii) theso-called on-line prediction, where the conditional probabilityP(x_{t+1}|...x_1x_2 ... x_t) (or the conditional densityP(x_{t+1}|x_1x_2 ... x_t)) should be estimated, where x_1x_2 ...x_t are given, iii) regression and iv) classification (or so-calledproblems with side information).We show that Kolmogorov complexity (KC) and universal codes (oruniversal data compressors), whose codeword length can beconsidered as an estimation of KC, can be used as a basis forconstructing asymptotically optimal methods for the above problems.(By definition, a universal code can "compress" any sequencegenerated by a stationary and ergodic source asymptotically to theShannon entropy of the source.)