Universal coding, information, prediction, and estimation

Authors:
J. Rissanen
Affiliations:
-
Venue:
IEEE Transactions on Information Theory
Year:
2006

Citing 0
Cited 16

Stochastic complexities of reduced rank regression in Bayesian estimation

Neural Networks
Prior knowledge guided maximum expected likelihood based model selection and adaptation for nonnative speech recognition

Computer Speech and Language
Multiple appearance models

Pattern Recognition
Recommending questions using the mdl-based tree cut model

Proceedings of the 17th international conference on World Wide Web
A comparative study of model selection criteria for computer vision applications

Image and Vision Computing
Parametric model-based motion segmentation using surface selection criterion

Computer Vision and Image Understanding
Nonlinear system identification via Laguerre network based fuzzy systems

Fuzzy Sets and Systems
Wavelet denoising with evolutionary algorithms

Digital Signal Processing
Scalable diagnosis in IP networks using path-based measurement and inference: A learning framework

Journal of Visual Communication and Image Representation
Context-based embedded image compression using binary wavelet transform

Image and Vision Computing
Spectral analysis of heart rate variability with the autoregressive method: What model order to choose?

Computers in Biology and Medicine
Learning coefficient of generalization error in bayesian estimation and vandermonde matrix-type singularity

Neural Computation
Applications of Kolmogorov Complexity and Universal Codes to Nonparametric Estimation of Characteristics of Time Series

Fundamenta Informaticae
A Common Algebraic Framework Of Empirical Learning Methods Based On Rough Sets And Matroid Theory

Fundamenta Informaticae
Lossless geometry compression for steady-state and time-varying irregular grids

EUROVIS'06 Proceedings of the Eighth Joint Eurographics / IEEE VGTC conference on Visualization
Search for sparse active inputs: a review

Information Theory, Combinatorics, and Search Theory

Quantified Score

Hi-index	754.84

Visualization

Abstract

A connection between universal codes and the problems of prediction and statistical estimation is established. A known lower bound for the mean length of universal codes is sharpened and generalized, and optimum universal codes constructed. The bound is defined to give the information in strings relative to the considered class of processes. The earlier derived minimum description length criterion for estimation of parameters, including their number, is given a fundamental information, theoretic justification by showing that its estimators achieve the information in the strings. It is also shown that one cannot do prediction in Gaussian autoregressive moving average (ARMA) processes below a bound, which is determined by the information in the data.