Probabilistic reasoning in intelligent systems: networks of plausible inference
Probabilistic reasoning in intelligent systems: networks of plausible inference
Generating functionology
Elements of information theory
Elements of information theory
Average Case Analysis of Algorithms on Sequences
Average Case Analysis of Algorithms on Sequences
Concrete Math
A fast normalized maximum likelihood algorithm for multinomial data
IJCAI'05 Proceedings of the 19th international joint conference on Artificial intelligence
Paper: Modeling by shortest data description
Automatica (Journal of IFAC)
Fisher information and stochastic complexity
IEEE Transactions on Information Theory
The minimum description length principle in coding and modeling
IEEE Transactions on Information Theory
Asymptotic minimax regret for data compression, gambling, and prediction
IEEE Transactions on Information Theory
Strong optimality of the normalized ML models as universal codes and information in data
IEEE Transactions on Information Theory
NML computation algorithms for tree-structured multinomial Bayesian networks
EURASIP Journal on Bioinformatics and Systems Biology
Decomposable Families of Itemsets
ECML PKDD '08 Proceedings of the European conference on Machine Learning and Knowledge Discovery in Databases - Part II
Fast NML computation for Naive Bayes models
DS'07 Proceedings of the 10th international conference on Discovery science
Learning locally minimax optimal Bayesian networks
International Journal of Approximate Reasoning
Detecting changes of clustering structures using normalized maximum likelihood coding
Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining
Using context and phonetic features in models of etymological sound change
EACL 2012 Proceedings of the EACL 2012 Joint Workshop of LINGVIS & UNCLH
| Hi-index | 0.89 |
The minimum description length (MDL) principle is a theoretically well-founded, general framework for performing model class selection and other types of statistical inference. This framework can be applied for tasks such as data clustering, density estimation and image denoising. The MDL principle is formalized via the so-called normalized maximum likelihood (NML) distribution, which has several desirable theoretical properties. The codelength of a given sample of data under the NML distribution is called the stochastic complexity, which is the basis for MDL model class selection. Unfortunately, in the case of discrete data, straightforward computation of the stochastic complexity requires exponential time with respect to the sample size, since the definition involves an exponential sum over all the possible data samples of a fixed size. As a main contribution of this paper, we derive an elegant recursion formula which allows efficient computation of the stochastic complexity in the case of n observations of a single multinomial random variable with K values. The time complexity of the new method is O(n+K) as opposed to O(nlognlogK) obtained with the previous results.