Universal coding for arbitrarily varying sources and for hierarchies of model classes

• Authors: N. Merhav
• Venue: DCC '95 Proceedings of the Conference on Data Compression
• Year: 1995

The minimum redundancy attainable by universal lossless codes for finite-state arbitrarily varying sources (AVS), and, in general, for an hierarchy of model classes, is investigated. In the AVS case, if the space of all possible underlying state sequences is partitioned into types, then the minimum universal coding redundancy can be essentially lower bounded by a quantity that decomposes into two terms, the first of which is the minimum redundancy within the type class (i.e., intra-type class redundancy), and the second is the minimum redundancy associated with a class of sources that can be thought of as "representatives" of the different types (i.e., inter-type class redundancy). This behavior can be generalized to universal coding for hierarchy of model classes, where each model class in the hierarchy has an increasing complexity. The lower bound for coding with respect to hierarchy of models is achievable by a Shannon code w.r.t an appropriate two-stage mixture, where the first stage mixture is over the sources in each class, and the second is a mixture over the indices of the model classes.