Generalized Dirichlet priors for Naïve Bayesian classifiers with multinomial models in document classification

Authors:
Tzu-Tsung Wong
Affiliations:
Institute of Information Management, National Cheng Kung University, Tainan, Taiwan, ROC 701
Venue:
Data Mining and Knowledge Discovery
Year:
2014

Citing 12
Cited 0

On estimating probabilities in tree pruning

EWSL-91 Proceedings of the European working session on learning on Machine learning
Generalized Dirichlet distribution in Bayesian analysis

Applied Mathematics and Computation
Machine Learning

Machine Learning
Naive (Bayes) at Forty: The Independence Assumption in Information Retrieval

ECML '98 Proceedings of the 10th European Conference on Machine Learning
A comparison of event models for Naive Bayes anti-spam e-mail filtering

EACL '03 Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics - Volume 1
Some Effective Techniques for Naive Bayes Text Classification

IEEE Transactions on Knowledge and Data Engineering
Raising the baseline for high-precision text classifiers

Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining
Discretization for naive-Bayes learning: managing discretization bias and variance

Machine Learning
Alternative prior assumptions for improving the performance of naïve Bayesian classifiers

Data Mining and Knowledge Discovery
The WEKA data mining software: an update

ACM SIGKDD Explorations Newsletter
Parameter estimation for generalized Dirichlet distributions from the sample estimates of the first and the second moments of random variables

Computational Statistics & Data Analysis
A hybrid discretization method for naïve Bayesian classifiers

Pattern Recognition

Quantified Score

Hi-index	0.00

Visualization

Abstract

The generalized Dirichlet distribution has been shown to be a more appropriate prior than the Dirichlet distribution for naïve Bayesian classifiers. When the dimension of a generalized Dirichlet random vector is large, the computational effort for calculating the expected value of a random variable can be high. In document classification, the number of distinct words that is the dimension of a prior for naïve Bayesian classifiers is generally more than ten thousand. Generalized Dirichlet priors can therefore be inapplicable for document classification from the viewpoint of computational efficiency. In this paper, some properties of the generalized Dirichlet distribution are established to accelerate the calculation of the expected values of random variables. Those properties are then used to construct noninformative generalized Dirichlet priors for naïve Bayesian classifiers with multinomial models. Our experimental results on two document sets show that generalized Dirichlet priors can achieve a significantly higher prediction accuracy and that the computational efficiency of naïve Bayesian classifiers is preserved.