Axiomatic characterization of the directed divergences and their linear combinations

  • Authors:
  • R. Johnson

  • Affiliations:
  • -

  • Venue:
  • IEEE Transactions on Information Theory
  • Year:
  • 1979

Quantified Score

Hi-index 754.84

Visualization

Abstract

The directed divergences of two probability densitiespandqare given byint p(x) log (p(x)/q(x))dxand by the same expression withpandqinterchanged; the divergence is the sum of the directed divergences. These quantities have applications in information theory and to the problem of assigning prior probabilities subject to constraints. It is shown that the directed divergences and their positive linear combinations, including the divegeoce, are characterized by axioms of it{positivity, additivity}, and it{finiteness}, which are fundamental in work on prior probabilities. In the course of the proof, the latter two are shown to imply yet another axiom: it{linear invariance}.