Applying Bayesian approach to decision tree

Authors:
Yatong Zhou;Taiyi Zhang;Zhigang Chen
Affiliations:
Dept. Information and Communication Engineering, Xi'an Jiaotong University, Xi'an, P.R. China;Dept. Information and Communication Engineering, Xi'an Jiaotong University, Xi'an, P.R. China;Dept. Information and Communication Engineering, Xi'an Jiaotong University, Xi'an, P.R. China
Venue:
ICIC'06 Proceedings of the 2006 international conference on Intelligent computing: Part II
Year:
2006

Citing 3
Cited 2

Bayesian MARS

Statistics and Computing
Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities

Machine Learning
Pattern Classification (2nd Edition)

Pattern Classification (2nd Edition)

Decision trees: a recent overview

Artificial Intelligence Review
A hybrid decision tree classifier

Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology

Quantified Score

Hi-index	0.00

Visualization

Abstract

Applying Bayesian approach to decision tree (DT) model, and then a Bayesian-inference-based decision tree (BDT) model is proposed. For BDT we assign prior to the model parameters. Together with observed samples, prior are converted to posterior through Bayesian inference. When making inference we resort to simulation methods using reversible jump Markov chain Monte Carlo (RJMCMC) since the dimension of posterior distribution is varying. Compared with DT, BDT enjoys the following three advantages. Firstly, the model's learning procedure is implemented with sampling instead of a series of splitting and pruning operations. Secondly, the model provides output that gives insight into different tree structures and recursive partition of the decision space, resulting in better classification accuracy. And thirdly, the model can indicate confidence that the sample belongs to a particular class in classification. The experiments on music style classification demonstrate the efficiency of BDT.