Redundancy of universal coding, Kolmogorov complexity, and Hausdorff dimension

  • Authors:

  • H. Takahashi

  • Affiliations:

  • Inst. of Stat. Math., Tokyo, Japan

  • Venue:
  • IEEE Transactions on Information Theory
  • Year:
  • 2006

Quantified Score

Hi-index 754.84

Visualization

Abstract

We study asymptotic code lengths of universal codes for parametric models. We show a universal code whose code length is asymptotically less than or equal to that of the minimum description length (MDL) code. Especially when some of the parameters of a source are not random reals, the coefficient of the logarithm in the formula of our universal code is less than that of the MDL code. We describe the redundancy in terms of Kolmogorov complexity and Hausdorff dimension. We show that our universal code is asymptotically optimal in the sense that the coefficient of the logarithm in the formula of the code length is minimal. Our universal code can be considered to be a natural extension of the Shannon code and the MDL code.