Discriminative versus generative parameter and structure learning of Bayesian network classifiers

Authors:
Franz Pernkopf;Jeff Bilmes
Affiliations:
Graz University of Technology, Inffeldgasse, Graz, Austria;University of Washington, Seattle, WA
Venue:
ICML '05 Proceedings of the 22nd international conference on Machine learning
Year:
2005

Citing 14
Cited 21

Probabilistic reasoning in intelligent systems: networks of plausible inference

Probabilistic reasoning in intelligent systems: networks of plausible inference
Elements of information theory

Elements of information theory
Numerical recipes in FORTRAN (2nd ed.): the art of scientific computing

Numerical recipes in FORTRAN (2nd ed.): the art of scientific computing
Knowledge representation and inference in similarity networks and Bayesian multinets

Artificial Intelligence
Wrappers for feature subset selection

Artificial Intelligence - Special issue on relevance
Bayesian Network Classifiers

Machine Learning - Special issue on learning with probabilistic representations
Machine Learning

Machine Learning
Probabilistic Networks and Expert Systems

Probabilistic Networks and Expert Systems
Dynamic Bayesian Multinets

UAI '00 Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Structural extension to logistic regression: discriminative parameter learning of belief net classifiers

Eighteenth national conference on Artificial intelligence
Natural statistical models for automatic speech recognition

Natural statistical models for automatic speech recognition
Discriminative, generative and imitative learning

Discriminative, generative and imitative learning
Learning Bayesian network classifiers by maximizing conditional likelihood

ICML '04 Proceedings of the twenty-first international conference on Machine learning
Bayesian network classifiers versus selective k-NN classifier

Pattern Recognition

A recursive method for discriminative mixture learning

Proceedings of the 24th international conference on Machine learning
Discriminative learning of Bayesian network classifiers

AIAP'07 Proceedings of the 25th conference on Proceedings of the 25th IASTED International Multi-Conference: artificial intelligence and applications
Boosted Bayesian network classifiers

Machine Learning
Discriminative Structure Learning of Markov Logic Networks

ILP '08 Proceedings of the 18th international conference on Inductive Logic Programming
A Discriminative Learning Method of TAN Classifier

ECSQARU '07 Proceedings of the 9th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Discriminative vs. Generative Learning of Bayesian Network Classifiers

ECSQARU '07 Proceedings of the 9th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty
Broad phonetic classification using discriminative Bayesian networks

Speech Communication
Predicting the location of mobile users: a machine learning approach

Proceedings of the 2009 international conference on Pervasive services
Competitive generative models with structure learning for NLP classification tasks

EMNLP '06 Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing
On Discriminative Parameter Learning of Bayesian Network Classifiers

ECML PKDD '09 Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II
Structured prediction by joint kernel support estimation

Machine Learning
Bayesian Network Structure Learning by Recursive Autonomy Identification

The Journal of Machine Learning Research
Feature selection for Bayesian network classifiers using the MDL-FS score

International Journal of Approximate Reasoning
Efficient Heuristics for Discriminative Structure Learning of Bayesian Network Classifiers

The Journal of Machine Learning Research
Sequence classification via large margin hidden Markov models

Data Mining and Knowledge Discovery
Efficient Structure Learning of Bayesian Networks using Constraints

The Journal of Machine Learning Research
Discriminative Learning of Bayesian Networks via Factorized Conditional Log-Likelihood

The Journal of Machine Learning Research
Bayesian learning of markov network structure

ECML'06 Proceedings of the 17th European conference on Machine Learning
Learning Bayesian network classifiers by risk minimization

International Journal of Approximate Reasoning
Graphical models as surrogates for complex ground motion models

ICAISC'12 Proceedings of the 11th international conference on Artificial Intelligence and Soft Computing - Volume Part II
Semi-supervised learning of hidden conditional random fields for time-series classification

Neurocomputing

Quantified Score

Hi-index	0.00

Visualization

Abstract

In this paper, we compare both discriminative and generative parameter learning on both discriminatively and generatively structured Bayesian network classifiers. We use either maximum likelihood (ML) or conditional maximum likelihood (CL) to optimize network parameters. For structure learning, we use either conditional mutual information (CMI), the explaining away residual (EAR), or the classification rate (CR) as objective functions. Experiments with the naive Bayes classifier (NB), the tree augmented naive Bayes classifier (TAN), and the Bayesian multinet have been performed on 25 data sets from the UCI repository (Merz et al., 1997) and from (Kohavi & John, 1997). Our empirical study suggests that discriminative structures learnt using CR produces the most accurate classifiers on almost half the data sets. This approach is feasible, however, only for rather small problems since it is computationally expensive. Discriminative parameter learning produces on average a better classifier than ML parameter learning.