Probabilistic reasoning in intelligent systems: networks of plausible inference
Probabilistic reasoning in intelligent systems: networks of plausible inference
Elements of information theory
Elements of information theory
Learning in graphical models
Bayesian Networks and Decision Graphs
Bayesian Networks and Decision Graphs
Probabilistic Networks and Expert Systems
Probabilistic Networks and Expert Systems
Constraint Processing
Graphs and Hypergraphs
Estimation and Marginalization Using the Kikuchi Approximation Methods
Neural Computation
Pattern Recognition and Machine Learning (Information Science and Statistics)
Pattern Recognition and Machine Learning (Information Science and Statistics)
Graphical Models, Exponential Families, and Variational Inference
Foundations and Trends® in Machine Learning
Constructing free-energy approximations and generalized belief propagation algorithms
IEEE Transactions on Information Theory
Hi-index | 0.00 |
We consider the problem of specifying the joint distribution of a collection of variables with maximum entropy when a set of marginals are fixed. One can easily derive that the structure of the solution joint distribution is that of a graphical model. The potential functions are then marginals at some power. We address the following question, Under which conditions on the set of constraints is it possible to fully identify the canonical exponents in the maximum entropy solution as functions of the problem structure? Literature related to this topic is somewhat scattered in disciplines such as statistical mechanics, information theory, graph theory, and inference in graphical models. In this article we gather and link results from these different fields. From this, we show that for a particular class of constraints set on marginals, the chordal maximal coherent sets of constraints, it is possible to derive the canonical exponents of the graphical model solution of the maximum entropy problem as the numbers of occurrences of separators in an associated join tree. Conversely, we present sufficient conditions to ensure that a graphical model is a solution of a maximum entropy problem.