Kullback-Leibler distance between probabilistic context-free grammars and probabilistic finite automata

Authors:
Mark-Jan Nederhof;Giorgio Satta
Affiliations:
University of Groningen, AS Groningen, The Netherlands;University of Padua, Padova, Italy
Venue:
COLING '04 Proceedings of the 20th international conference on Computational Linguistics
Year:
2004

Citing 7
Cited 4

Computation of Probabilities for an Island-Driven Parser

IEEE Transactions on Pattern Analysis and Machine Intelligence
Probabilistic Languages: A Review and Some Open Questions

ACM Computing Surveys (CSUR)
Introduction To Automata Theory, Languages, And Computation

Introduction To Automata Theory, Languages, And Computation
Practical experiments with regular approximation of context-free languages

Computational Linguistics - Special issue on finite-state methods in NLP
Finite-state transducers in language and speech processing

Computational Linguistics
The recognition capacity of local syntactic constraints

EACL '91 Proceedings of the fifth conference on European chapter of the Association for Computational Linguistics
Precise n-gram probabilities from stochastic context-free grammars

ACL '94 Proceedings of the 32nd annual meeting on Association for Computational Linguistics

Cross-entropy and estimation of probabilistic context-free grammars

HLT-NAACL '06 Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics
Probabilistic Context-Free Grammars Estimated from Infinite Distributions

IEEE Transactions on Pattern Analysis and Machine Intelligence
Sentence-level contextual opinion retrieval

Proceedings of the 20th international conference companion on World wide web
Approximate inference: A sampling based modeling technique to capture complex dependencies in a language model

Speech Communication

Quantified Score

Hi-index	0.00

Visualization

Abstract

We consider the problem of computing the Kullback-Leibler distance, also called the relative entropy, between a probabilistic context-free grammar and a probabilistic finite automaton. We show that there is a closed-form (analytical) solution for one part of the Kullback-Leibler distance, viz. the cross-entropy. We discuss several applications of the result to the problem of distributional approximation of probabilistic context-free grammars by means of probabilistic finite automata.