Estimating a probability using finite memory
IEEE Transactions on Information Theory
Elements of information theory
Elements of information theory
Probability estimation for the Q-Coder
IBM Journal of Research and Development - Q-Coder adaptive binary arithmetic coder
Statistical estimation with bounded memory
Statistics and Computing
| Hi-index | 0.00 |
In this work we consider the problem of universal sequential probability assignment, under the self-information loss, where the machine for performing the universal probability assignment is constrained to have a finite memory. Sequential probability assignment is equivalent to lossless source coding if we ignore the number of states required to convert the probability estimate into code bits. We consider both the probabilistic setting where the sequence is generated by a probabilistic source (either Bernoulli i.i.d. source or q-th order Markov source), and the deterministic setting where the sequence is a deterministic individual sequence.We also consider the case where the universal machine is deterministic, randomized, time-invariant or time-variant. We provide in most cases lower bounds and describe finite memory universal machines whose performance, in terms of the memory size, is compared to these bounds.