Fast NML computation for Naive Bayes models

Authors:
Tommi Mononen;Petri Myllymäki
Affiliations:
Complex Systems Computation Group, Helsinki Institute for Information Technology, University of Helsinki & Helsinki University of Technology and Department of Computer Science, University of Helsi ...;Complex Systems Computation Group, Helsinki Institute for Information Technology, University of Helsinki & Helsinki University of Technology and Department of Computer Science, University of Helsi ...
Venue:
DS'07 Proceedings of the 10th international conference on Discovery science
Year:
2007

Citing 12
Cited 0

Probabilistic reasoning in intelligent systems: networks of plausible inference

Probabilistic reasoning in intelligent systems: networks of plausible inference
Learning Bayesian Networks: The Combination of Knowledge and Statistical Data

Machine Learning
The art of computer programming, volume 2 (3rd ed.): seminumerical algorithms

The art of computer programming, volume 2 (3rd ed.): seminumerical algorithms
Automatic Computations with Power Series

Journal of the ACM (JACM)
Stochastic Complexity in Statistical Inquiry Theory

Stochastic Complexity in Statistical Inquiry Theory
Advances in Minimum Description Length: Theory and Applications (Neural Information Processing)

Advances in Minimum Description Length: Theory and Applications (Neural Information Processing)
The Minimum Description Length Principle (Adaptive Computation and Machine Learning)

The Minimum Description Length Principle (Adaptive Computation and Machine Learning)
A linear-time algorithm for computing the multinomial stochastic complexity

Information Processing Letters
NML computation algorithms for tree-structured multinomial Bayesian networks

EURASIP Journal on Bioinformatics and Systems Biology
Information and Complexity in Statistical Modeling

Information and Complexity in Statistical Modeling
Analytic Combinatorics

Analytic Combinatorics
Fisher information and stochastic complexity

IEEE Transactions on Information Theory

Quantified Score

Hi-index	0.00

Visualization

Abstract

The Minimum Description Length (MDL) is an informationtheoretic principle that can be used for model selection and other statistical inference tasks. One way to implement this principle in practice is to compute the Normalized Maximum Likelihood (NML) distribution for a given parametric model class. Unfortunately this is a computationally infeasible task for many model classes of practical importance. In this paper we present a fast algorithm for computing the NML for the Naive Bayes model class, which is frequently used in classification and clustering tasks. The algorithm is based on a relationship between powers of generating functions and discrete convolution. The resulting algorithm has the time complexity of O(n2), where n is the size of the data.