A fast normalized maximum likelihood algorithm for multinomial data

Authors:
Petri Kontkanen;Petri Myllymäki
Affiliations:
Complex Systems Computation Group, Helsinki Institute for Information Technology, University of Helsinki & Helsinki University of Technology, Finland;Complex Systems Computation Group, Helsinki Institute for Information Technology, University of Helsinki & Helsinki University of Technology, Finland
Venue:
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Year:
2005

Citing 10
Cited 4

On predictive distributions and Bayesian networks

Statistics and Computing
On Bayesian Case Matching

EWCBR '98 Proceedings of the 4th European Workshop on Advances in Case-Based Reasoning
The Minimum Description Length Principle (Adaptive Computation and Machine Learning)

The Minimum Description Length Principle (Adaptive Computation and Machine Learning)
Supervised model-based visualization of high-dimensional data

Intelligent Data Analysis
Minimum encoding approaches for predictive modeling

UAI'98 Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence
Fisher information and stochastic complexity

IEEE Transactions on Information Theory
Universal prediction

IEEE Transactions on Information Theory
The minimum description length principle in coding and modeling

IEEE Transactions on Information Theory
Asymptotic minimax regret for data compression, gambling, and prediction

IEEE Transactions on Information Theory
Strong optimality of the normalized ML models as universal codes and information in data

IEEE Transactions on Information Theory

A linear-time algorithm for computing the multinomial stochastic complexity

Information Processing Letters
NML computation algorithms for tree-structured multinomial Bayesian networks

EURASIP Journal on Bioinformatics and Systems Biology
Combining initial segments of lists

ALT'11 Proceedings of the 22nd international conference on Algorithmic learning theory
Combining initial segments of lists

Theoretical Computer Science

Quantified Score

Hi-index	0.00

Visualization

Abstract

Stochastic complexity of a data set is defined as the shortest possible code length for the data obtainable by using some fixed set of models. This measure is of great theoretical and practical importance as a tool for tasks such as model selection or data clustering. In the case of multinomial data, computing the modern version of stochastic complexity, defined as the Normalized Maximum Likelihood (NML) criterion, requires computing a sum with an exponential number of terms. Furthermore, in order to apply NML in practice, one often needs to compute a whole table of these exponential sums. In our previous work, we were able to compute this table by a recursive algorithm. The purpose of this paper is to significantly improve the time complexity of this algorithm. The techniques used here are based on the discrete Fourier transformand the convolution theorem.