Parameter estimation for probabilistic finite-state transducers

Authors:
Jason Eisner
Affiliations:
Johns Hopkins University, Baltimore, MD
Venue:
ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Year:
2002

Citing 17
Cited 47

Rational series and their languages

Rational series and their languages
Inducing Features of Random Fields

IEEE Transactions on Pattern Analysis and Machine Intelligence
Iterative methods for solving linear systems

Iterative methods for solving linear systems
Hidden Markov models with finite state supervision

Extended finite state models of language
A Unified Approach to Path Problems

Journal of the ACM (JACM)
Fast Algorithms for Solving Path Problems

Journal of the ACM (JACM)
Finite-State Language Processing

Finite-State Language Processing
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Maximum Entropy Markov Models for Information Extraction and Segmentation

ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
A Rational Design for a Weighted Finite-State Transducer Library

WIA '97 Revised Papers from the Second International Workshop on Implementing Automata
An Extendible Regular Expression Compiler for Finite-State Approaches in Natural Language Processing

WIA '99 Revised Papers from the 4th International Workshop on Automata Implementation
Smoothing a probablistic lexicon via syntactic transformations

Smoothing a probablistic lexicon via syntactic transformations
Practical experiments with regular approximation of context-free languages

Computational Linguistics - Special issue on finite-state methods in NLP
Machine transliteration

Computational Linguistics
Transducers from rewrite rules with backreferences

EACL '99 Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics
Compilation of weighted finite-state transducers from decision trees

ACL '96 Proceedings of the 34th annual meeting on Association for Computational Linguistics
An efficient compiler for weighted rewrite rules

ACL '96 Proceedings of the 34th annual meeting on Association for Computational Linguistics

Trust evaluation in ad-hoc networks

Proceedings of the 3rd ACM workshop on Wireless security
Probabilistic Finite-State Machines-Part II

IEEE Transactions on Pattern Analysis and Machine Intelligence
Simpler and more general minimization for weighted finite-state automata

NAACL '03 Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - Volume 1
An interactive spreadsheet for teaching the forward-backward algorithm

ETMTNLP '02 Proceedings of the ACL-02 Workshop on Effective tools and methodologies for teaching natural language processing and computational linguistics - Volume 1
Transformational priors over grammars

EMNLP '02 Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10
Statistical machine translation by parsing

ACL '04 Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics
Dyna: a declarative language for implementing dynamic programs

ACLdemo '04 Proceedings of the ACL 2004 on Interactive poster and demonstration sessions
Contrastive estimation: training log-linear models on unlabeled data

ACL '05 Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics
Compiling Comp Ling: practical weighted dynamic programming and the Dyna language

HLT '05 Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
Learning stochastic edit distance: Application in handwritten character recognition

Pattern Recognition
Case-factor diagrams for structured probabilistic modeling

Journal of Computer and System Sciences
Propagating multitrust within trust networks

Proceedings of the 2008 ACM symposium on Applied computing
Training tree transducers

Computational Linguistics
Book review:

Computational Linguistics
Answering questions with authority

Proceedings of the 17th ACM conference on Information and knowledge management
SCLP for Trust Propagation in Small-World Networks

Recent Advances in Constraints
Cube summing, approximate inference with non-local features, and dynamic programming without semirings

EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Translation as weighted deduction

EACL '09 Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics
Latent-variable modeling of string transductions with finite-state methods

EMNLP '08 Proceedings of the Conference on Empirical Methods in Natural Language Processing
A stochastic finite-state morphological parser for Turkish

ACLShort '09 Proceedings of the ACL-IJCNLP 2009 Conference Short Papers
Variational decoding for statistical machine translation

ACL '09 Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2
First- and second-order expectation semirings with applications to minimum-risk training on translation forests

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
Graphical models over multiple strings

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
Consensus training for consensus decoding in machine translation

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3 - Volume 3
Model Checking for a Class of Weighted Automata

Discrete Event Dynamic Systems
Learning state machine-based string edit kernels

Pattern Recognition
Rewriting the orthography of sms messages

Natural Language Engineering
Finding cognate groups using phylogenies

ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
Phylogenetic grammar induction

ACL '10 Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics
Efficient path counting transducers for minimum bayes-risk decoding of statistical machine translation lattices

ACLShort '10 Proceedings of the ACL 2010 Conference Short Papers
Joshua 2.0: a toolkit for parsing-based machine translation with syntax, semirings, discriminative training and other goodies

WMT '10 Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
Automated whole sentence grammar correction using a noisy channel model

HLT '11 Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1
Reestimation of reified rules in semiring parsing and biparsing

SSST-5 Proceedings of the Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation
A spectral learning algorithm for finite state transducers

ECML PKDD'11 Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part I
Bilingual random walk models for automated grammar correction of ESL author-produced text

IUNLPBEA '11 Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications
Using learned conditional distributions as edit distance

SSPR'06/SPR'06 Proceedings of the 2006 joint IAPR international conference on Structural, Syntactic, and Statistical Pattern Recognition
A discriminative model of stochastic edit distance in the form of a conditional transducer

ICGI'06 Proceedings of the 8th international conference on Grammatical Inference: algorithms and applications
A class of rational n-WFSM auto-intersections

CIAA'05 Proceedings of the 10th international conference on Implementation and Application of Automata
Dyna: extending datalog for modern AI

Datalog'10 Proceedings of the First international conference on Datalog Reloaded
The Latent Maximum Entropy Principle

ACM Transactions on Knowledge Discovery from Data (TKDD)
Gradient computation in linear-chain conditional random fields using the entropy message passing algorithm

Pattern Recognition Letters
Hope and fear for discriminative training of statistical translation models

The Journal of Machine Learning Research
Unsupervised learning on an approximate corpus

NAACL HLT '12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Stanford: probabilistic edit distance metrics for STS

SemEval '12 Proceedings of the First Joint Conference on Lexical and Computational Semantics - Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation
Exploiting partial annotations with EM training

WILS '12 Proceedings of the NAACL-HLT Workshop on the Induction of Linguistic Structure
Probabilistic finite state machines for regression-based MT evaluation

EMNLP-CoNLL '12 Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning
SPEDE: probabilistic edit distance metrics for MT evaluation

WMT '12 Proceedings of the Seventh Workshop on Statistical Machine Translation

Quantified Score

Hi-index	0.00

Visualization

Abstract

Weighted finite-state transducers suffer from the lack of a training algorithm. Training is even harder for transducers that have been assembled via finite-state operations such as composition, minimization, union, concatenation, and closure, as this yields tricky parameter tying. We formulate a "parameterized FST" paradigm and give training algorithms for it, including a general bookkeeping trick ("expectation semirings") that cleanly and efficiently computes expectations and gradients.