A general minimax result for relative entropy

  • Authors:
  • D. Haussler

  • Affiliations:
  • Dept. of Comput. & Inf. Sci., California Univ., Santa Cruz, CA

  • Venue:
  • IEEE Transactions on Information Theory
  • Year:
  • 2006

Quantified Score

Hi-index 754.90

Visualization

Abstract

Suppose nature picks a probability measure Pθ on a complete separable metric space X at random from a measurable set P Θ={Pθ:θ∈Θ}. Then, without knowing θ, a statistician picks a measure Q on S. Finally, the statistician suffers a loss D(P0||Q), the relative entropy between Pθ and Q. We show that the minimax and maximin values of this game are always equal, and there is always a minimax strategy in the closure of the set of all Bayes strategies. This generalizes previous results of Gallager(1979), and Davisson and Leon-Garcia (1980)