A large-deviation analysis for the maximum likelihood learning of tree structures

Authors:
Vincent Y. F. Tan;Animashree Anandkumar;Lang Tong;Alan S. Willsky
Affiliations:
Stochastic Systems Group, LIDS, MIT, Cambridge, MA;Stochastic Systems Group, LIDS, MIT, Cambridge, MA and ECE Dept., Cornell University, Ithaca, NY;ECE Dept., Cornell University, Ithaca, NY;Stochastic Systems Group, LIDS, MIT, Cambridge, MA
Venue:
ISIT'09 Proceedings of the 2009 IEEE international conference on Symposium on Information Theory - Volume 2
Year:
2009

Citing 1
Cited 4

Introduction to Algorithms

Introduction to Algorithms

How do the structure and the parameters of Gaussian tree models affect structure learning?

Allerton'09 Proceedings of the 47th annual Allerton conference on Communication, control, and computing
Learning Gaussian tree models: analysis of error exponents and extremal structures

IEEE Transactions on Signal Processing
Learning graphical models for hypothesis testing and classification

IEEE Transactions on Signal Processing
Scalable multi-dimensional user intent identification using tree structured distributions

Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval

Quantified Score

Hi-index	0.01

Visualization

Abstract

The problem of maximum-likelihood learning of the structure of an unknown discrete distribution from samples is considered when the distribution is Markov on a tree. Large-deviation analysis of the error in estimation of the set of edges of the tree is performed. Necessary and sufficient conditions are provided to ensure that this error probability decays exponentially. These conditions are based on the mutual information between each pair of variables being distinct from that of other pairs. The rate of error decay, or error exponent, is derived using the large-deviation principle. The error exponent is approximated using Euclidean information theory and is given by a ratio, to be interpreted as the signal-to-noise ratio (SNR) for learning. Numerical experiments show the SNR approximation is accurate.