On the expected codeword length per symbol of optimal prefix codes for extended sources

• Authors: Jay Cheng
• Venue: IWCMC '07 Proceedings of the 2007 international conference on Wireless communications and mobile computing
• Year: 2007

For a discrete memoryless source X, it is well known that the expected codeword length per symbol Ln(X) of an optimal prefix code for the extended source Xn converges to the source entropy. However, the sequence Ln(X) need not be nonincreasing. As the encoding and decoding complexity increases exponentially with the block length, from a practical perspective it is both important and of interest to know when we can have a decrease in the expected codeword length per symbol by increasing the block length. In this paper, we obtain some results on the behavior of Ln(X) and provide sufficient conditions for Lkn(X)Ln(X) in terms of k, n, the probability of the most likely source symbol p1 and/or the minimum codeword length of an optimal code for the original source. By using these sufficient conditions, for any given n ≥ 1 and non-dyadic pn1, we also obtain an integer k* ≥ 2 such that Lk'n(X) Ln(X) for all k' ≥ k*. Our results could be regarded as extensions and generalizations of those by Montgomery and Kumar.