Probability Density Estimation by Perturbing and Combining Tree Structured Markov Networks

Authors:
Sourour Ammar;Philippe Leray;Boris Defourny;Louis Wehenkel
Affiliations:
Knowledge and Decision Team, Laboratoire d'Informatique de Nantes Atlantique (LINA) UMR 6241, Ecole Polytechnique de l'Université de Nantes, France;Knowledge and Decision Team, Laboratoire d'Informatique de Nantes Atlantique (LINA) UMR 6241, Ecole Polytechnique de l'Université de Nantes, France;Department of Electrical Engineering and Computer Science & GIGA-Research, University of Lièèège, Belgium;Department of Electrical Engineering and Computer Science & GIGA-Research, University of Lièèège, Belgium
Venue:
ECSQARU '09 Proceedings of the 10th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Year:
2009

Citing 10
Cited 2

Fusion, propagation, and structuring in belief networks

Artificial Intelligence
Bagging predictors

Machine Learning
Efficient Approximations for the MarginalLikelihood of Bayesian Networks with Hidden Variables

Machine Learning - Special issue on learning with probabilistic representations
Probabilistic Networks and Expert Systems

Probabilistic Networks and Expert Systems
Looking for lumps: boosting and bagging for density estimation

Computational Statistics & Data Analysis - Nonlinear methods and data mining
Being Bayesian about Network Structure

UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Learning with mixtures of trees

Learning with mixtures of trees
The Cross Entropy Method: A Unified Approach To Combinatorial Optimization, Monte-carlo Simulation (Information Science and Statistics)

The Cross Entropy Method: A Unified Approach To Combinatorial Optimization, Monte-carlo Simulation (Information Science and Statistics)
Naive Bayes models for probability estimation

ICML '05 Proceedings of the 22nd international conference on Machine learning
Extremely randomized trees

Machine Learning

Mixture of Markov trees for Bayesian network structure learning with small datasets in high dimensional space

ECSQARU'11 Proceedings of the 11th European conference on Symbolic and quantitative approaches to reasoning with uncertainty
Efficiently approximating Markov tree bagging for high-dimensional density estimation

ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part III

Quantified Score

Hi-index	0.00

Visualization

Abstract

To explore the Perturb and Combine idea for estimating probability densities, we study mixtures of tree structured Markov networks derived by bagging combined with the Chow and Liu maximum weight spanning tree algorithm, or by pure random sampling. We empirically assess the performances of these methods in terms of accuracy, with respect to mixture models derived by EM-based learning of Naive Bayes models, and EM-based learning of mixtures of trees. We find that the bagged ensembles outperform all other methods while the random ones perform also very well. Since the computational complexity of the former is quadratic and that of the latter is linear in the number of variables of interest, this paves the way towards the design of efficient density estimation methods that may be applied to problems with very large numbers of variables and comparatively very small sample sizes.