Superior Guarantees for Sequential Prediction and Lossless Compression via Alphabet Decomposition
The Journal of Machine Learning Research
Reliability criteria in information theory and in statistical hypothesis testing
Foundations and Trends in Communications and Information Theory
Fisher information determinant and stochastic complexity for Markov models
ISIT'09 Proceedings of the 2009 IEEE international conference on Symposium on Information Theory - Volume 3
Tunstall code, Khodak variations, and random walks
IEEE Transactions on Information Theory
| Hi-index | 754.90 |
Redundancy of universal codes for a class of sources determines by how much the actual code length exceeds the optimal code length. In the minimax scenario, one designs the best code for the worst source within the class. Such minimax redundancy comes in two flavors: average minimax or worst case minimax. We study the worst case minimax redundancy of universal block codes for Markovian sources of any order. We prove that the maximal minimax redundancy for Markov sources of order r is asymptotically equal to 1/2mr(m-1)log2n+log2Amr-(lnlnm1(m-1)/)/lnm+o(1), where n is the length of a source sequence, m is the size of the alphabet, and Amr is an explicit constant (e.g., we find that for a binary alphabet m=2 and Markov of order r=1 the constant A21=16·G≈14.655449504 where G is the Catalan number). Unlike previous attempts, we view the redundancy problem as an asymptotic evaluation of certain sums over a set of matrices representing Markov types. The enumeration of Markov types is accomplished by reducing it to counting Eulerian paths in a multigraph. In particular, we propose exact and asymptotic formulas for the number of strings of a given Markov type. All of these findings are obtained by analytic and combinatorial tools of analysis of algorithms.