Information, Divergence and Risk for Binary Experiments
The Journal of Machine Learning Research
The Journal of Machine Learning Research
| Hi-index | 754.86 |
The concept of efficiency measures of multicategory information systems as well as the concepts of the relationship and the similarity measures between two efficiency measures have been recently introduced and developed. In this paper, the concave measures as a general class of efficiency measures and the information measures as a special class of concave measures are defined and investigated. The relationship between any concave measure, information measure in particular, and the Bayes probability of errorP_{B}is determined for any2leq Q < infty, whereQdenotes the number of categories in a multicategory information system. The so-calledepsilon_{ }0 andepsilon_{m}criteria are proposed as the similarity measures between the information measures andP_{B}. The problems of determination, for any2 leq Q < infty, of all the information measures with minimalepsilon_{0}andepsilon_{m}criteria, calledepsilon_{0}-optimal andepsilon_{m}-optimal, are formulated and completely solved, respectively. Additionally, the minimal values ofepsilon_{0}andepsilon_{m}criteria are evaluated as well. It is pointed out that the well-known average conditional quadratic entropy is for all2 leq Q < inftyvery close to theepsilon_{0}-optimal andepsilon_{m}-Optimal information measures with respect toepsilon_{0}andepsilon_{m}criteria, respectively.