Introduction to algorithms
Computation of Probabilities for an Island-Driven Parser
IEEE Transactions on Pattern Analysis and Machine Intelligence
Elements of information theory
Elements of information theory
IEEE Transactions on Pattern Analysis and Machine Intelligence
Handbook of formal languages, vol. 3
Statistical methods for speech recognition
Statistical methods for speech recognition
Foundations of statistical natural language processing
Foundations of statistical natural language processing
Statistical Language Learning
Introduction To Automata Theory, Languages, And Computation
Introduction To Automata Theory, Languages, And Computation
Pattern Classification (2nd Edition)
Pattern Classification (2nd Edition)
Computation of the probability of initial substring generation by stochastic context-free grammars
Computational Linguistics
Probabilistic top-down parsing and language modeling
Computational Linguistics
Estimation of probabilistic context-free grammars
Computational Linguistics
Statistical properties of probabilistic context-free grammars
Computational Linguistics
Exploiting syntactic structure for language modeling
COLING '98 Proceedings of the 17th international conference on Computational linguistics - Volume 1
Probabilistic Finite-State Machines-Part II
IEEE Transactions on Pattern Analysis and Machine Intelligence
Probabilistic Finite-State Machines-Part I
IEEE Transactions on Pattern Analysis and Machine Intelligence
Immediate-head parsing for language models
ACL '01 Proceedings of the 39th Annual Meeting on Association for Computational Linguistics
A General Technique to Train Language Models on Language Models
Computational Linguistics
Contrastive estimation: training log-linear models on unlabeled data
ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Applying Probability Measures to Abstract Languages
IEEE Transactions on Computers
Solution of an Open Problem on Probabilistic Grammars
IEEE Transactions on Computers
Recursive markov chains, stochastic grammars, and monotone systems of nonlinear equations
STACS'05 Proceedings of the 22nd annual conference on Theoretical Aspects of Computer Science
Maximum likelihood analysis of algorithms and data structures
Theoretical Computer Science
| Hi-index | 0.14 |
In this paper, we consider probabilistic context-free grammars, a class of generative devices that has been successfully exploited in several applications of syntactic pattern matching, especially in statistical natural language parsing. We investigate the problem of training probabilistic context-free grammars on the basis of distributions defined over an infinite set of trees or an infinite set of sentences by minimizing the cross-entropy. This problem has applications in cases of context-free approximation of distributions generated by more expressive statistical models. We show several interesting theoretical properties of probabilistic context-free grammars that are estimated in this way, including the previously unknown equivalence between the grammar cross-entropy with the input distribution and the so-called derivational entropy of the grammar itself. We discuss important consequences of these results involving the standard application of the maximum-likelihood estimator on finite tree and sentence samples, as well as other finite-state models such as Hidden Markov Models and probabilistic finite automata.