A lower bound on compression of unknown alphabets
Theoretical Computer Science
Coding on countably infinite alphabets
IEEE Transactions on Information Theory
Connections between probability estimation and graph theory
Allerton'09 Proceedings of the 47th annual Allerton conference on Communication, control, and computing
On information divergence measures and a unified typicality
IEEE Transactions on Information Theory
| Hi-index | 754.96 |
We study the redundancy of three approaches to compression of independent and identically distributed (i.i.d.) strings over large, possibly infinite, alphabets: standard compression of the string itself and compression of the string's shape and pattern, which describe its symbols' relative magnitude and precedence, respectively. We determine the rate at which per-symbol standard redundancy increases to infinity as the alphabet size increases, show that the maximum per-symbol shape redundancy is between 0.027 and 1, and compare these to results showing that per-symbol pattern redundancy diminishes to zero for all alphabet sizes. We relate these concepts to ordered and unordered partitions of integers and sets, and use this framework to explore relations between several combinatorial quantities, including the Bell, Fubini, and second-type Stirling numbers.