Elements of information theory
Elements of information theory
Predictability, Complexity, and Learning
Neural Computation
IEEE Transactions on Information Theory
On uniqueness Theorems for Tsallis entropy and Tsallis relative entropy
IEEE Transactions on Information Theory
| Hi-index | 0.00 |
The theoretical basis for a candidate variational principle for the information bottleneck (IB) method is formulated within the ambit of the generalized nonadditive statistics of Tsallis. Given a nonadditivity parameter q, the role of the additive duality of nonadditive statistics (q* = 2 - q) in relating Tsallis entropies for ranges of the nonadditivity parameter q q 1 is described. Defining X, X, and Y to be the source alphabet, the compressed reproduction alphabet, and, the relevance variable respectively, it is demonstrated that minimization of a generalized IB (gIB) Lagrangian defined in terms of the nonadditivity parameter q* self-consistently yields the nonadditive effective distortion measure to be the q-deformed generalized Kullback-Leibler divergence: DK-Lq[p(Y|X)||p(Y|X)]. This result is achieved without enforcing any a-priori assumptions. Next, it is proven that the q*-deformed nonadditive free energy of the system is non-negative and convex. Finally, the update equations for the gIB method are derived. These results generalize critical features of the IB method to the case of Tsallis statistics.