Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing)
Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing)
Information and Complexity in Statistical Modeling
Information and Complexity in Statistical Modeling
A Kullback-Leibler View of Linear and Log-Linear Pools
Decision Analysis
International Journal of Approximate Reasoning
A general minimax result for relative entropy
IEEE Transactions on Information Theory
A new geometric capacity characterization of a discrete memoryless channel
IEEE Transactions on Information Theory
A source matching approach to finding minimax codes
IEEE Transactions on Information Theory
IEEE Transactions on Information Theory
The uniform distribution as a universal prior
IEEE Transactions on Information Theory
Parametric Estimation of the Local False Discovery Rate for Identifying Genetic Associations
IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB)
Hi-index | 0.00 |
In the typical analysis of a data set, a single method is selected for statistical reporting even when equally applicable methods yield very different results. Examples of equally applicable methods can correspond to those of different ancillary statistics in frequentist inference and of different prior distributions in Bayesian inference. More broadly, choices are made between parametric and nonparametric methods and between frequentist and Bayesian methods. Rather than choosing a single method, it can be safer, in a game-theoretic sense, to combine those that are equally appropriate in light of the available information. Since methods of combining subjectively assessed probability distributions are not objective enough for that purpose, this paper introduces a method of distribution combination that does not require any assignment of distribution weights. It does so by formalizing a hedging strategy in terms of a game between three players: nature, a statistician combining distributions, and a statistician refusing to combine distributions. The optimal move of the first statistician reduces to the solution of a simpler problem of selecting an estimating distribution that minimizes the Kullback-Leibler loss maximized over the plausible distributions to be combined. The resulting combined distribution is a linear combination of the most extreme of the distributions to be combined that are scientifically plausible. The optimal weights are close enough to each other that no extreme distribution dominates the others. The new methodology is illustrated by combining conflicting empirical Bayes methods in the context of gene expression data analysis.