Noisy constrained capacity for BSC channels

• Authors: Philippe Jacquet;Wojciech Szpankowski
• Venue: IEEE Transactions on Information Theory
• Year: 2010

We study the classical problem of noisy constrained capacity in the case of the binary symmetric channel (BSC), namely, the capacity of a BSC whose input is a sequence from a constrained set. As stated by Fan et al., "... while calculation of the noise-free capacity of constrained sequences is well known, the computation of the capacity of a constraint in the presence of noise... has been an unsolved problem in the half-century since Shannon's landmark paper." We first express the constrained capacity of a binary symmetric channel with (d, k-constrained input as a limit of the top Lyapunov exponents of certain matrix random processes. Then, we compute asymptotic approximations of the noisy constrained capacity for cases where the noise parameter ε is small. In particular, we show that when K≤2d, the error term (excess of capacity beyond the noise-free capacity) is O(ε), whereas it is O(ε log ε) when k 2d. In both cases, we compute the coefficient of the error term. In the course of establishing these findings, we also extend our previous results on the entropy of a hidden Markov process to higher-order finite memory processes. These conclusions are proved by a combination of analytic and combinatorial methods.