Probabilistic reasoning in intelligent systems: networks of plausible inference
Probabilistic reasoning in intelligent systems: networks of plausible inference
Elements of information theory
Elements of information theory
A revolution: belief propagation in graphs with cycles
NIPS '97 Proceedings of the 1997 conference on Advances in neural information processing systems 10
Graph Algorithms
Using Mutual Information to Determine Relevance in Bayesian Networks
PRICAI '98 Proceedings of the 5th Pacific Rim International Conference on Artificial Intelligence: Topics in Artificial Intelligence
Algorithm Design
Survey propagation: An algorithm for satisfiability
Random Structures & Algorithms
An edge deletion semantics for belief propagation and its practical impact on approximation quality
AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Iterative join-graph propagation
UAI'02 Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence
A comparative study of energy minimization methods for markov random fields
ECCV'06 Proceedings of the 9th European conference on Computer Vision - Volume Part II
Tree-based reparameterization framework for analysis of sum-product and related algorithms
IEEE Transactions on Information Theory
Constructing free-energy approximations and generalized belief propagation algorithms
IEEE Transactions on Information Theory
Turbo decoding as an instance of Pearl's “belief propagation” algorithm
IEEE Journal on Selected Areas in Communications
Relax, compensate and then recover
JSAI-isAI'10 Proceedings of the 2010 international conference on New Frontiers in Artificial Intelligence
| Hi-index | 0.00 |
We consider the problem of computing mutual information between many pairs of variables in a Bayesian network. This task is relevant to a new class of Generalized Belief Propagation (GBP) algorithms that characterizes Iterative Belief Propagation (IBP) as a polytree approximation found by deleting edges in a Bayesian network. By computing, in the simplified network, the mutual information between variables across a deleted edge, we can estimate the impact that recovering the edge might have on the approximation. Unfortunately, it is computationally impractical to compute such scores for networks over many variables having large state spaces. So that edge recovery can scale to such networks, we propose in this paper an approximation of mutual information which is based on a soft extension of d-separation (a graphical test of independence in Bayesian networks). We focus primarily on polytree networks, which are sufficient for the application we consider, although we discuss potential extensions of the approximation to general networks as well. Empirically, we show that our proposal is often as effective as mutual information for the task of edge recovery, with orders of magnitude savings in computation time in larger networks. Our results lead to a concrete realization of GBP, admitting improvements to IBP approximations with only a modest amount of computational effort.