Elements of information theory
Elements of information theory
Error control systems for digital communication and storage
Error control systems for digital communication and storage
High-performance communication networks
High-performance communication networks
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
Probabilistic Networks and Expert Systems
Probabilistic Networks and Expert Systems
FOCS '02 Proceedings of the 43rd Symposium on Foundations of Computer Science
Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation
UAI '01 Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence
Good Codes Based on Very Sparse Matrices
Proceedings of the 5th IMA Conference on Cryptography and Coding
Low complexity, high performance algorithms for estimation and decoding
Low complexity, high performance algorithms for estimation and decoding
Correctness of Local Probability Propagation in Graphical Models with Loops
Neural Computation
The geometry of turbo-decoding dynamics
IEEE Transactions on Information Theory
The generalized distributive law
IEEE Transactions on Information Theory
The capacity of low-density parity-check codes under message-passing decoding
IEEE Transactions on Information Theory
Design of capacity-approaching irregular low-density parity-check codes
IEEE Transactions on Information Theory
A new look at the generalized distributive law
IEEE Transactions on Information Theory
Turbo decoding as an instance of Pearl's “belief propagation” algorithm
IEEE Journal on Selected Areas in Communications
Graphical Models, Exponential Families, and Variational Inference
Foundations and Trends® in Machine Learning
Convexity arguments for efficient minimization of the Bethe and Kikuchi free energies
Journal of Artificial Intelligence Research
Spine detection and labeling using a parts-based graphical model
IPMI'07 Proceedings of the 20th international conference on Information processing in medical imaging
Convexifying the Bethe free energy
UAI '09 Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence
SIAM Journal on Discrete Mathematics
Decomposition and Approximation of Loopy Bayesian Networks
Fundamenta Informaticae
| Hi-index | 0.00 |
In this letter, we examine a general method of approximation, known as the Kikuchi approximation method, for finding the marginals of a product distribution, as well as the corresponding partition function. The Kikuchi approximation method defines a certain constrained optimization problem, called the Kikuchi problem, and treats its stationary points as approximations to the desired marginals. We show how to associate a graph to any Kikuchi problem and describe a class of local message-passing algorithms along the edges of any such graph, which attempt to find the solutions to the problem. Implementation of these algorithms on graphs with fewer edges requires fewer operations in each iteration. We therefore characterize minimal graphs for a Kikuchi problem, which are those with the minimum number of edges. We show with empirical results that these simpler algorithms often offer significant savings in computational complexity, without suffering a loss in the convergence rate. We give conditions for the convexity of a given Kikuchi problem and the exactness of the approximations in terms of the loops of the minimal graph. More precisely, we show that if the minimal graph is cycle free, then the Kikuchi approximation method is exact, and the converse is also true generically. Together with the fact that in the cycle-free case, the iterative algorithms are equivalent to the well-known belief propagation algorithm, our results imply that, generically, the Kikuchi approximation method can be exact if and only if traditional junction tree methods could also solve the problem exactly.