Probabilistic reasoning in intelligent systems: networks of plausible inference
Probabilistic reasoning in intelligent systems: networks of plausible inference
On Compatible Priors for Bayesian Networks
IEEE Transactions on Pattern Analysis and Machine Intelligence
Bayesian classification (AutoClass): theory and results
Advances in knowledge discovery and data mining
Machine Learning - Special issue on learning with probabilistic representations
Stochastic Complexity in Statistical Inquiry Theory
Stochastic Complexity in Statistical Inquiry Theory
On predictive distributions and Bayesian networks
Statistics and Computing
Minimum encoding approaches for predictive modeling
UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Models and selection criteria for regression and classification
UAI'97 Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence
Fisher information and stochastic complexity
IEEE Transactions on Information Theory
IEEE Transactions on Information Theory
The minimum description length principle in coding and modeling
IEEE Transactions on Information Theory
Unsupervised Bayesian visualization of high-dimensional data
Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining
An Unsupervised Bayesian Distance Measure
EWCBR '00 Proceedings of the 5th European Workshop on Advances in Case-Based Reasoning
Adaptive Bayesian network classifiers
Intelligent Data Analysis
Discriminative model selection for belief net structures
AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Bayesian Network Structure Learning by Recursive Autonomy Identification
The Journal of Machine Learning Research
Classifier learning with supervised marginal likelihood
UAI'01 Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence
Bayesian class-matched multinet classifier
SSPR'06/SPR'06 Proceedings of the 2006 joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition
Learning Bayesian network classifiers by risk minimization
International Journal of Approximate Reasoning
Bias management of bayesian network classifiers
DS'05 Proceedings of the 8th international conference on Discovery Science
Credal ensembles of classifiers
Computational Statistics & Data Analysis
| Hi-index | 0.00 |
Given a set of possible models (e.g., Bayesian network structures) and a data sample, in the unsupervised model selection problem the task is to choose the most accurate model with respect to the domain joint probability distribution. In contrast to this, in supervised model selection it is a priori known that the chosen model will be used in the future for prediction tasks involving more "focused" predictive distributions. Although focused predictive distributions can be produced from the joint probability distribution by marginalization, in practice the best model in the unsupervised sense does not necessarily perform well in supervised domains. In particular, the standard marginal likelihood score is a criterion for the unsupervised task, and, although frequently used for supervised model selection also, does not perform well in such tasks. In this paper we study the performance of the marginal likelihood score empirically in supervised Bayesian network selection tasks by using a large number of publicly available classification data sets, and compare the results to those obtained by alternative model selection criteria, including empirical crossvalidation methods, an approximation of a supervised marginal likelihood measure, and a supervised version of Dawid's prequential (predictive sequential) principle. The results demonstrate that the marginal likelihood score does not perform well for supervised model selection, while the best results are obtained by using Dawid's prequential approach.