Some inequalities relating different measures of divergence between two probability distributions

  • Authors:
  • L. Withers, Jr.

  • Affiliations:
  • Naval Res. Lab., Washington, DC

  • Venue:
  • IEEE Transactions on Information Theory
  • Year:
  • 2006

Quantified Score

Hi-index 754.90

Visualization

Abstract

This note presents new inequalities relating different divergence measures in the family of “convex likelihood-ratio expectation” measures of Csiszar (1967), Ali and Silvey (1966), and especially in the single-parameter family of “AM-GM” divergence measures. The most prominent result is that θ2 ⩽¼J, where θ is the Bhattacharyya angle of divergence (a true distance metric), and J is the symmetric cross-entropy. A pair of “log Γ” divergences is also introduced and related to the cross-entropies I and J