Gradient computation in linear-chain conditional random fields using the entropy message passing algorithm

Authors:
Velimir M. Ilić;Dejan I. MančEv;Branimir T. Todorović;Miomir S. Stanković
Affiliations:
Mathematical Institute of the Serbian Academy of Sciences and Arts, Kneza Mihaila 36, 11000 Beograd, Serbia and University of Niš, Faculty of Sciences and Mathematics, Višegradska 33, 18 ...;University of Niš, Faculty of Sciences and Mathematics, Višegradska 33, 18000 Niš, Serbia;University of Niš, Faculty of Sciences and Mathematics, Višegradska 33, 18000 Niš, Serbia;University of Niš, Faculty of Occupational Safety, arnojevića 10a, 18000 Niš, Serbia
Venue:
Pattern Recognition Letters
Year:
2012

Citing 14
Cited 0

Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

ICML '01 Proceedings of the Eighteenth International Conference on Machine Learning
Machine learning techniques for the computer security domain of anomaly detection

Machine learning techniques for the computer security domain of anomaly detection
Parameter estimation for probabilistic finite-state transducers

ACL '02 Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Exploring the Use of Conditional Random Field Models and HMMs for Historical Handwritten Document Recognition

DIAL '06 Proceedings of the Second International Conference on Document Image Analysis for Libraries
Accelerated training of conditional random fields with stochastic gradient methods

ICML '06 Proceedings of the 23rd international conference on Machine learning
Conditional Random Fields for Intrusion Detection

AINAW '07 Proceedings of the 21st International Conference on Advanced Information Networking and Applications Workshops - Volume 01
Efficient training methods for conditional random fields

Efficient training methods for conditional random fields
On the memory complexity of the forward-backward algorithm

Pattern Recognition Letters
First- and second-order expectation semirings with applications to minimum-risk training on translation forests

EMNLP '09 Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 1 - Volume 1
The generalized distributive law

IEEE Transactions on Information Theory
Factor graphs and the sum-product algorithm

IEEE Transactions on Information Theory
On receiver structures for channels having memory

IEEE Transactions on Information Theory
Optimal decoding of linear codes for minimizing symbol error rate (Corresp.)

IEEE Transactions on Information Theory
Entropy Message Passing

IEEE Transactions on Information Theory

Quantified Score

Hi-index	0.10

Visualization

Abstract

The paper proposes a numerically stable recursive algorithm for the exact computation of the linear-chain conditional random field gradient. It operates as a forward algorithm over the log-domain expectation semiring and has the purpose of enhancing memory efficiency when applied to long observation sequences. Unlike the traditional algorithm based on the forward-backward recursions, the memory complexity of our algorithm does not depend on the sequence length. The experiments on real data show that it can be useful for the problems which deal with long sequences.